[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:07 dist-test-slave-pgkr krb5kdc[19762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:07 dist-test-slave-pgkr krb5kdc[19762](info): set up 2 sockets
Aug 06 22:42:07 dist-test-slave-pgkr krb5kdc[19762](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 22:42:08.720250 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:08 dist-test-slave-pgkr krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:08.839859 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:44789
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:44789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:08.921707 19775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:08.921882 19775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:08.921902 19775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:08.923087 19775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:08.923130 19775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:08.923143 19775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:08.923156 19775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:08.923202 19775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:08.924765 19775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:44789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:44789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:08.924985 19775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:08.925195 19775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:08.930115 19780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:08.930143 19781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:08.930116 19783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:08.930271 19775 server_base.cc:1034] running on GCE node
I20260806 22:42:08.930553 19775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:08.930889 19775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:08.932060 19775 hybrid_clock.cc:648] HybridClock initialized: now 1786056128932061 us; error 105 us; skew 500 ppm
Aug 06 22:42:08 dist-test-slave-pgkr krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056128, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:08.934499 19775 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:08.935030 19775 webserver.cc:460] Webserver started at http://127.19.74.254:42055/ using document root <none> and password file <none>
I20260806 22:42:08.935256 19775 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:08.935322 19775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:08.935478 19775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:08.936501 19775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "7eac293551bb45e99eef34b2968eb970"
format_stamp: "Formatted at 2026-08-06 22:42:08 on dist-test-slave-pgkr"
server_key: "3498b776978fc4f5ee426954e748fbb4"
server_key_iv: "12bbd56982b6b11bd27655ca557e0185"
server_key_version: "encryptionkey@0"
I20260806 22:42:08.936874 19775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "7eac293551bb45e99eef34b2968eb970"
format_stamp: "Formatted at 2026-08-06 22:42:08 on dist-test-slave-pgkr"
server_key: "3498b776978fc4f5ee426954e748fbb4"
server_key_iv: "12bbd56982b6b11bd27655ca557e0185"
server_key_version: "encryptionkey@0"
I20260806 22:42:08.938165 19775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:08.938946 19790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:08.939117 19775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:08.939232 19775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "7eac293551bb45e99eef34b2968eb970"
format_stamp: "Formatted at 2026-08-06 22:42:08 on dist-test-slave-pgkr"
server_key: "3498b776978fc4f5ee426954e748fbb4"
server_key_iv: "12bbd56982b6b11bd27655ca557e0185"
server_key_version: "encryptionkey@0"
I20260806 22:42:08.939316 19775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:08.975255 19775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:08.977367 19775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:08.977538 19775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:08.981685 19775 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:44789
I20260806 22:42:08.981722 19844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:44789 every 8 connection(s)
I20260806 22:42:08.982090 19775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:08.982995 19845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:08.985319 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 19775
I20260806 22:42:08.985337 19845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970: Bootstrap starting.
I20260806 22:42:08.985448 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:08.985982 19845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:08.986105 19755 external_mini_cluster.cc:1428] Setting key 1eb29d5cbda5eedfc468437ecd62d19e
I20260806 22:42:08.986313 19845 log.cc:824] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:08.987162 19845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970: No bootstrap required, opened a new log
I20260806 22:42:08.989583 19845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac293551bb45e99eef34b2968eb970" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44789 } }
I20260806 22:42:08.989802 19845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:08.989854 19845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eac293551bb45e99eef34b2968eb970, State: Initialized, Role: FOLLOWER
I20260806 22:42:08.989989 19845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac293551bb45e99eef34b2968eb970" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44789 } }
I20260806 22:42:08.990096 19845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:08.990129 19845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:08.990211 19845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:08.991469 19845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac293551bb45e99eef34b2968eb970" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44789 } }
I20260806 22:42:08.991587 19845 leader_election.cc:302] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eac293551bb45e99eef34b2968eb970; no voters:
I20260806 22:42:08.991796 19845 leader_election.cc:288] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:08.991860 19850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:08.992019 19850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [term 1 LEADER]: Becoming Leader. State: Replica: 7eac293551bb45e99eef34b2968eb970, State: Running, Role: LEADER
I20260806 22:42:08.992064 19845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:08.992182 19850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac293551bb45e99eef34b2968eb970" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44789 } }
I20260806 22:42:08.992623 19851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eac293551bb45e99eef34b2968eb970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac293551bb45e99eef34b2968eb970" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44789 } } }
I20260806 22:42:08.992703 19851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:08.992885 19852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eac293551bb45e99eef34b2968eb970. Latest consensus state: current_term: 1 leader_uuid: "7eac293551bb45e99eef34b2968eb970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac293551bb45e99eef34b2968eb970" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44789 } } }
I20260806 22:42:08.992940 19852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:08.993366 19859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:08.993959 19859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:08.996426 19859 catalog_manager.cc:1347] Generated new cluster ID: 7e448803f9644d7694eead5caff59f65
I20260806 22:42:08.996526 19859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:42:08 dist-test-slave-pgkr krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:09.002933 19859 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:09.003630 19859 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:09.006078 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:08.986999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.19.74.254:44789)
0806 22:42:08.987198 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:08.987207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:08.987447 (+ 240us) server_negotiation.cc:407] Connection header received
0806 22:42:08.989756 (+ 2309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:08.989761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:08.989798 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:08.990128 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:08.992703 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:08.994091 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:08.995628 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:08.996281 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:08.996396 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.001560 (+ 5164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:09.001564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:09.001564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:09.001577 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:09.004041 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.004541 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.004544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.004545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.004658 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.004962 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.004962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.004963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.005212 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:09.005374 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:09.005700 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:42:09.005840 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260806 22:42:09.008657 19859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eac293551bb45e99eef34b2968eb970: Generated new TSK 0
I20260806 22:42:09.008826 19859 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:09.037341 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44789
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:09.119259 19872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:09.119494 19872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:09.119534 19872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:09.120656 19872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:09.120725 19872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:09.120770 19872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:09.122225 19872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:09.122493 19872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:09.122752 19872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:09.127779 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:09.127859 19872 server_base.cc:1034] running on GCE node
W20260806 22:42:09.127763 19881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:09.127756 19877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:09.128103 19872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:09.128376 19872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:09.129521 19872 hybrid_clock.cc:648] HybridClock initialized: now 1786056129129513 us; error 28 us; skew 500 ppm
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:09.131990 19872 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:09.132473 19872 webserver.cc:460] Webserver started at http://127.19.74.193:36007/ using document root <none> and password file <none>
I20260806 22:42:09.132720 19872 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:09.132789 19872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:09.132941 19872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:09.133980 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "83a0cd2480e545abbf80d17f8d1d71b3"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "703ea033a4b91a4e70e4361dd24281e6"
server_key_iv: "5306cfd7fb2b34bcf2b94514c008cb15"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.134349 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "83a0cd2480e545abbf80d17f8d1d71b3"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "703ea033a4b91a4e70e4361dd24281e6"
server_key_iv: "5306cfd7fb2b34bcf2b94514c008cb15"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.135735 19872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:09.136464 19888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.136636 19872 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:09.136739 19872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "83a0cd2480e545abbf80d17f8d1d71b3"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "703ea033a4b91a4e70e4361dd24281e6"
server_key_iv: "5306cfd7fb2b34bcf2b94514c008cb15"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.136830 19872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:09.144263 19872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:09.145931 19872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:09.146088 19872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:09.146306 19872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:09.146602 19872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:09.146731 19872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.146770 19872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:09.146801 19872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.152717 19872 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:42843
I20260806 22:42:09.152770 20005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:42843 every 8 connection(s)
I20260806 22:42:09.153056 19872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:09.162122 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:09.153673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:45587 (local address 127.19.74.254:44789)
0806 22:42:09.153724 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:09.153726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:09.153963 (+ 237us) server_negotiation.cc:407] Connection header received
0806 22:42:09.154489 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:09.154491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:09.154504 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:09.154559 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:09.155659 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.156171 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.157193 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.157406 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.157495 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.159494 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:09.159495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:09.159495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:09.159503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:09.160711 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.161128 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.161129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.161130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.161193 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.161459 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.161460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.161460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.161594 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:09.161665 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:09.161933 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 22:42:09.162030 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:42:09.162276 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:09.153655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:44789 (local address 127.19.74.193:45587)
0806 22:42:09.153845 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:09.153879 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:09.154372 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:09.154667 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:09.154672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:09.155357 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:09.155538 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:09.155542 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.156268 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.156269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:09.157064 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:09.157066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.157509 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.157510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:09.157634 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.158066 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:09.158074 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:09.159381 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:09.160797 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:09.160804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:09.160807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:09.161024 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:09.161263 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:09.161263 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:09.161264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:09.161378 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:09.161667 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:09.161675 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:09.161806 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 22:42:09.161908 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:42:09.162050 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260806 22:42:09.162842 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 19872
I20260806 22:42:09.162936 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:09.162919 20006 heartbeater.cc:347] Connected to a master server at 127.19.74.254:44789
I20260806 22:42:09.163014 20006 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:09.163095 19755 external_mini_cluster.cc:1428] Setting key 5a148a198e9330645ace1c37f868abcc
I20260806 22:42:09.163208 20006 heartbeater.cc:511] Master 127.19.74.254:44789 requested a full tablet report, sending...
I20260806 22:42:09.163795 19807 ts_manager.cc:194] Registered new tserver with Master: 83a0cd2480e545abbf80d17f8d1d71b3 (127.19.74.193:42843)
I20260806 22:42:09.164395 19807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:45587
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:09.187433 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44789
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:09.271996 20012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:09.272198 20012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:09.272222 20012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:09.273346 20012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:09.273382 20012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:09.273408 20012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:09.274919 20012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:09.275127 20012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:09.275296 20012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:09.280437 20020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:09.280515 20012 server_base.cc:1034] running on GCE node
W20260806 22:42:09.280475 20018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:09.280460 20017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:09.280849 20012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:09.281086 20012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:09.282244 20012 hybrid_clock.cc:648] HybridClock initialized: now 1786056129282238 us; error 37 us; skew 500 ppm
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:09.284787 20012 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:09.285288 20012 webserver.cc:460] Webserver started at http://127.19.74.194:36689/ using document root <none> and password file <none>
I20260806 22:42:09.285537 20012 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:09.285604 20012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:09.285756 20012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:09.286816 20012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "912519f0a4144e02aeabac9503b119d8"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "c8dacb3a6a0939a84b22a329155c0be5"
server_key_iv: "af199fbb4b303326cd5030b9e1510e0d"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.287173 20012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "912519f0a4144e02aeabac9503b119d8"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "c8dacb3a6a0939a84b22a329155c0be5"
server_key_iv: "af199fbb4b303326cd5030b9e1510e0d"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.288455 20012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:09.289095 20027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.289259 20012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:09.289361 20012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "912519f0a4144e02aeabac9503b119d8"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "c8dacb3a6a0939a84b22a329155c0be5"
server_key_iv: "af199fbb4b303326cd5030b9e1510e0d"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.289429 20012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:09.300609 20012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:09.302361 20012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:09.302523 20012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:09.302757 20012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:09.303079 20012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:09.303134 20012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.303167 20012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:09.303192 20012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.309357 20012 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:35233
I20260806 22:42:09.309403 20143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:35233 every 8 connection(s)
I20260806 22:42:09.309700 20012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:09.312064 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 20012
I20260806 22:42:09.312168 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:09.312374 19755 external_mini_cluster.cc:1428] Setting key e2f0e11040231382610889033f7621cf
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:09.319531 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:09.310306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:59923 (local address 127.19.74.254:44789)
0806 22:42:09.310375 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:09.310376 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:09.310666 (+ 290us) server_negotiation.cc:407] Connection header received
0806 22:42:09.311129 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:09.311130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:09.311141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:09.311203 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:09.312274 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.312824 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.313837 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.314029 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.314107 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.316602 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:09.316603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:09.316603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:09.316619 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:09.318005 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.318514 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.318515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.318515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.318565 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.318907 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.318908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.318908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.319008 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:09.319069 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:09.319409 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 22:42:09.319469 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:42:09.319833 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:09.310291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:44789 (local address 127.19.74.194:59923)
0806 22:42:09.310452 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:09.310498 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:09.311016 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:09.311257 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:09.311260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:09.311807 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:09.312117 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:09.312124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.312925 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.312926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:09.313725 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:09.313727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.314129 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.314130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:09.314252 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.314834 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:09.314844 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:09.316516 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:09.318087 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:09.318095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:09.318097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:09.318430 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:09.318652 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:09.318653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:09.318672 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:09.318825 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:09.319085 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:09.319092 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:09.319262 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:42:09.319380 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:42:09.319552 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260806 22:42:09.320346 20144 heartbeater.cc:347] Connected to a master server at 127.19.74.254:44789
I20260806 22:42:09.320451 20144 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:09.320642 20144 heartbeater.cc:511] Master 127.19.74.254:44789 requested a full tablet report, sending...
I20260806 22:42:09.321039 19807 ts_manager.cc:194] Registered new tserver with Master: 912519f0a4144e02aeabac9503b119d8 (127.19.74.194:35233)
I20260806 22:42:09.321390 19807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:59923
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:09.339520 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44789
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:09.422597 20149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:09.422820 20149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:09.422844 20149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:09.424026 20149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:09.424068 20149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:09.424130 20149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:09.425616 20149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:09.425858 20149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:09.426115 20149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:09.431051 20154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:09.431053 20155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:09.431053 20157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:09.431257 20149 server_base.cc:1034] running on GCE node
I20260806 22:42:09.431485 20149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:09.431737 20149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:09.432907 20149 hybrid_clock.cc:648] HybridClock initialized: now 1786056129432900 us; error 48 us; skew 500 ppm
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:09.435539 20149 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:09.436007 20149 webserver.cc:460] Webserver started at http://127.19.74.195:43927/ using document root <none> and password file <none>
I20260806 22:42:09.436262 20149 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:09.436331 20149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:09.436472 20149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:09.437533 20149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "af3ce8606ab5490dbf31ed587272a52b"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "767925ab2b4e51f8c4c052a0aa835c58"
server_key_iv: "452035e1564b2f86db8df5e89a50979a"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.437898 20149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "af3ce8606ab5490dbf31ed587272a52b"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "767925ab2b4e51f8c4c052a0aa835c58"
server_key_iv: "452035e1564b2f86db8df5e89a50979a"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.439282 20149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:42:09.440050 20165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.440204 20149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:09.440325 20149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "af3ce8606ab5490dbf31ed587272a52b"
format_stamp: "Formatted at 2026-08-06 22:42:09 on dist-test-slave-pgkr"
server_key: "767925ab2b4e51f8c4c052a0aa835c58"
server_key_iv: "452035e1564b2f86db8df5e89a50979a"
server_key_version: "encryptionkey@0"
I20260806 22:42:09.440419 20149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:09.451901 20149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:09.453683 20149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:09.453843 20149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:09.454064 20149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:09.454376 20149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:09.454432 20149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.454474 20149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:09.454502 20149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:09.460525 20149 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:45269
I20260806 22:42:09.460577 20282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:45269 every 8 connection(s)
I20260806 22:42:09.460884 20149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:09.464962 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 20149
I20260806 22:42:09.465066 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:09.465255 19755 external_mini_cluster.cc:1428] Setting key 5c530f8101647bd2eeea788a80a97672
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:09.469707 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:09.461632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:38539 (local address 127.19.74.254:44789)
0806 22:42:09.461680 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:09.461682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:09.461871 (+ 189us) server_negotiation.cc:407] Connection header received
0806 22:42:09.462343 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:09.462345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:09.462355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:09.462406 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:09.463307 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.463800 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.464731 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.464913 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.464963 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.467417 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:09.467418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:09.467418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:09.467425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:09.468564 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.468924 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.468924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.468925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.468974 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.469193 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.469194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.469194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.469290 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:09.469340 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:09.469562 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 22:42:09.469633 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:09.469849 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:09.461629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:44789 (local address 127.19.74.195:38539)
0806 22:42:09.461771 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:09.461804 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:09.462257 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:09.462459 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:09.462462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:09.463032 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:09.463228 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:09.463233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.463886 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.463887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:09.464655 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:09.464657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.464996 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.464997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:09.465163 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.465840 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:09.465848 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:09.467335 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:09.468648 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:09.468654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:09.468655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:09.468861 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:09.469030 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:09.469031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:09.469031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:09.469132 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:09.469343 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:09.469347 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:09.469454 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 22:42:09.469534 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:42:09.469650 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260806 22:42:09.470373 20283 heartbeater.cc:347] Connected to a master server at 127.19.74.254:44789
I20260806 22:42:09.470451 20283 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:09.470708 20283 heartbeater.cc:511] Master 127.19.74.254:44789 requested a full tablet report, sending...
I20260806 22:42:09.471077 19807 ts_manager.cc:194] Registered new tserver with Master: af3ce8606ab5490dbf31ed587272a52b (127.19.74.195:45269)
I20260806 22:42:09.471408 19807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:38539
I20260806 22:42:09.477344 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:09.501554 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:09.494213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.19.74.193:42843)
0806 22:42:09.494382 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:09.494388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:09.494406 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:42:09.494434 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:09.494436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:09.494449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:09.494517 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:09.495133 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.495837 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.496579 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.496915 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.496983 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.499229 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:09.499238 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:09.499238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:09.499245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:09.500585 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.500902 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.500905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.500905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.500960 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.501119 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.501119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.501119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.501248 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:09.501357 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:09.501409 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:42:09.501473 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
W20260806 22:42:09.501977 19917 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41640
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:09.524406 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:09.517957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.19.74.193:42843)
0806 22:42:09.518041 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:09.518043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:09.518114 (+ 71us) server_negotiation.cc:407] Connection header received
0806 22:42:09.518156 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:09.518157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:09.518167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:09.518213 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:09.518744 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.519220 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.519884 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:09.520065 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:09.520120 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:09.522201 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:09.522202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:09.522203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:09.522209 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:09.523437 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.523736 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.523736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.523737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.523808 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:09.523984 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:09.523984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:09.523985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:09.524073 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:09.524172 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:09.524238 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:42:09.524299 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:42:09.524959 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 19872
I20260806 22:42:09.530946 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 20012
I20260806 22:42:09.537307 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 20149
I20260806 22:42:09.542445 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 19775
2026-08-06T22:42:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1893 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[20297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[20297](info): set up 2 sockets
Aug 06 22:42:09 dist-test-slave-pgkr krb5kdc[20297](info): commencing operation
krb5kdc: starting...
W20260806 22:42:10.632732 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:10 dist-test-slave-pgkr krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:10.752935 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:43959
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:43959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:10.834750 20310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:10.834987 20310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:10.835024 20310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:10.836153 20310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:10.836220 20310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:10.836243 20310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:10.836273 20310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:10.836318 20310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:10.837731 20310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:43959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:43959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:10.837998 20310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:10.838217 20310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:10.842942 20316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:10.842931 20318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:10.842931 20315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:10.843145 20310 server_base.cc:1034] running on GCE node
I20260806 22:42:10.843405 20310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:10.843720 20310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:10.844889 20310 hybrid_clock.cc:648] HybridClock initialized: now 1786056130844865 us; error 54 us; skew 500 ppm
Aug 06 22:42:10 dist-test-slave-pgkr krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056130, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:10.847337 20310 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:10.847837 20310 webserver.cc:460] Webserver started at http://127.19.74.254:34133/ using document root <none> and password file <none>
I20260806 22:42:10.848098 20310 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:10.848186 20310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:10.848331 20310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:10.849334 20310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "e0b451d3b12a4b29b8ce5310095d88f1"
format_stamp: "Formatted at 2026-08-06 22:42:10 on dist-test-slave-pgkr"
server_key: "53e7b08fa71be01c62824fe0a5ef04d1"
server_key_iv: "8da67722d3f517169e47a0601768ca05"
server_key_version: "encryptionkey@0"
I20260806 22:42:10.849697 20310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "e0b451d3b12a4b29b8ce5310095d88f1"
format_stamp: "Formatted at 2026-08-06 22:42:10 on dist-test-slave-pgkr"
server_key: "53e7b08fa71be01c62824fe0a5ef04d1"
server_key_iv: "8da67722d3f517169e47a0601768ca05"
server_key_version: "encryptionkey@0"
I20260806 22:42:10.851172 20310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:42:10.851970 20326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:10.852170 20310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:10.852294 20310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "e0b451d3b12a4b29b8ce5310095d88f1"
format_stamp: "Formatted at 2026-08-06 22:42:10 on dist-test-slave-pgkr"
server_key: "53e7b08fa71be01c62824fe0a5ef04d1"
server_key_iv: "8da67722d3f517169e47a0601768ca05"
server_key_version: "encryptionkey@0"
I20260806 22:42:10.852370 20310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:10.875028 20310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:10.876953 20310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:10.877125 20310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:10.881449 20310 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:43959
I20260806 22:42:10.881491 20380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:43959 every 8 connection(s)
I20260806 22:42:10.881865 20310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:10.882855 20381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:10.885200 20381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1: Bootstrap starting.
I20260806 22:42:10.885793 20381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:10.886080 20381 log.cc:824] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:10.886854 20381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1: No bootstrap required, opened a new log
I20260806 22:42:10.888180 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 20310
I20260806 22:42:10.888270 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:10.888454 19755 external_mini_cluster.cc:1428] Setting key 79cd9aa58d31ca3648a865ca8fc52efb
I20260806 22:42:10.888733 20381 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b451d3b12a4b29b8ce5310095d88f1" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43959 } }
I20260806 22:42:10.888944 20381 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:10.888993 20381 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b451d3b12a4b29b8ce5310095d88f1, State: Initialized, Role: FOLLOWER
I20260806 22:42:10.889145 20381 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b451d3b12a4b29b8ce5310095d88f1" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43959 } }
I20260806 22:42:10.889240 20381 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:10.889274 20381 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:10.889312 20381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:10.890023 20381 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b451d3b12a4b29b8ce5310095d88f1" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43959 } }
I20260806 22:42:10.890143 20381 leader_election.cc:302] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0b451d3b12a4b29b8ce5310095d88f1; no voters:
I20260806 22:42:10.890303 20381 leader_election.cc:288] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:10.890370 20386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:10.890528 20381 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:10.890514 20386 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [term 1 LEADER]: Becoming Leader. State: Replica: e0b451d3b12a4b29b8ce5310095d88f1, State: Running, Role: LEADER
I20260806 22:42:10.890734 20386 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b451d3b12a4b29b8ce5310095d88f1" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43959 } }
I20260806 22:42:10.891139 20386 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0b451d3b12a4b29b8ce5310095d88f1. Latest consensus state: current_term: 1 leader_uuid: "e0b451d3b12a4b29b8ce5310095d88f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b451d3b12a4b29b8ce5310095d88f1" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43959 } } }
I20260806 22:42:10.891229 20386 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:10.891149 20388 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0b451d3b12a4b29b8ce5310095d88f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b451d3b12a4b29b8ce5310095d88f1" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43959 } } }
I20260806 22:42:10.891443 20388 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:10.891572 20395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:10.892125 20395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:10.893766 20395 catalog_manager.cc:1347] Generated new cluster ID: 53ff2d48351240fb8d9884599396cb63
I20260806 22:42:10.893816 20395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:42:10 dist-test-slave-pgkr krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:10.901131 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:10.888990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46984 (local address 127.19.74.254:43959)
0806 22:42:10.889185 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:10.889195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:10.889230 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:42:10.889356 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:10.889364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:10.889419 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:10.889565 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:10.890144 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:10.891536 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:10.892616 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:10.893098 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:10.893219 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:10.896970 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:10.896975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:10.896975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:10.896998 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:10.899293 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:10.899733 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:10.899737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:10.899737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:10.899878 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:10.900122 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:10.900123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:10.900123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:10.900444 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:10.900638 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:10.900739 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:42:10.900884 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260806 22:42:10.915061 20395 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:10.915792 20395 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:10.921693 20395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0b451d3b12a4b29b8ce5310095d88f1: Generated new TSK 0
I20260806 22:42:10.921887 20395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:10.950887 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43959
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:11.033859 20407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:11.034135 20407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:11.034176 20407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:11.035411 20407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:11.035475 20407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:11.035529 20407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:11.037026 20407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43959
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:11.037302 20407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:11.037532 20407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:11.042539 20412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:11.042550 20415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:11.042587 20413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:11.042795 20407 server_base.cc:1034] running on GCE node
I20260806 22:42:11.043007 20407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:11.043286 20407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:11.044464 20407 hybrid_clock.cc:648] HybridClock initialized: now 1786056131044451 us; error 36 us; skew 500 ppm
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:11.047123 20407 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:11.047598 20407 webserver.cc:460] Webserver started at http://127.19.74.193:39867/ using document root <none> and password file <none>
I20260806 22:42:11.047834 20407 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:11.047912 20407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:11.048063 20407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:11.049099 20407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "571db821521d434eacaa666bafa02f55"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "bd60323e1057e63efcda8ea6761bb245"
server_key_iv: "00c86bc6d33e20105c1861e771fe5694"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.049474 20407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "571db821521d434eacaa666bafa02f55"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "bd60323e1057e63efcda8ea6761bb245"
server_key_iv: "00c86bc6d33e20105c1861e771fe5694"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.050959 20407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:11.051702 20422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.051877 20407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:11.051986 20407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "571db821521d434eacaa666bafa02f55"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "bd60323e1057e63efcda8ea6761bb245"
server_key_iv: "00c86bc6d33e20105c1861e771fe5694"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.052060 20407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:11.075845 20407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:11.077682 20407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:11.077876 20407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:11.078116 20407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:11.078438 20407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:11.078492 20407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.078534 20407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:11.078562 20407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.084694 20407 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:41743
I20260806 22:42:11.084749 20542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:41743 every 8 connection(s)
I20260806 22:42:11.085048 20407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:11.086067 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 20407
I20260806 22:42:11.086169 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:11.086330 19755 external_mini_cluster.cc:1428] Setting key 974a18143a7dcc14d6f0a48c5c31986f
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:11.095359 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.085749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:56601 (local address 127.19.74.254:43959)
0806 22:42:11.085813 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.085816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.086034 (+ 218us) server_negotiation.cc:407] Connection header received
0806 22:42:11.087287 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.087288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.087299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.087332 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:11.088193 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.088734 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.089708 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.090026 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.090079 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.092345 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:11.092346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:11.092346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:11.092354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:11.093785 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.094287 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.094288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.094289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.094353 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.094645 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.094646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.094646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.094764 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:11.094839 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.095127 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.095239 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:42:11.095527 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.085726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:43959 (local address 127.19.74.193:56601)
0806 22:42:11.085880 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:11.085941 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:11.087185 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:11.087425 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:11.087429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:11.087831 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:11.088096 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.088102 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.088845 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.088846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.089622 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.089624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.090099 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.090100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.090202 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.090742 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:11.090750 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:11.092188 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:11.093910 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:11.093917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:11.093919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:11.094191 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:11.094451 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:11.094452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:11.094452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:11.094554 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:11.094842 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:11.094850 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:11.094986 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 22:42:11.095086 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:42:11.095258 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
I20260806 22:42:11.096115 20543 heartbeater.cc:347] Connected to a master server at 127.19.74.254:43959
I20260806 22:42:11.096228 20543 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:11.096413 20543 heartbeater.cc:511] Master 127.19.74.254:43959 requested a full tablet report, sending...
I20260806 22:42:11.096926 20344 ts_manager.cc:194] Registered new tserver with Master: 571db821521d434eacaa666bafa02f55 (127.19.74.193:41743)
I20260806 22:42:11.097508 20344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:56601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:11.111694 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43959
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:11.195163 20548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:11.195361 20548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:11.195385 20548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:11.196553 20548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:11.196594 20548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:11.196620 20548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:11.198019 20548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43959
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:11.198216 20548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:11.198383 20548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:11.203449 20553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:11.203523 20554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:11.203435 20556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:11.203684 20548 server_base.cc:1034] running on GCE node
I20260806 22:42:11.203923 20548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:11.204169 20548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:11.205312 20548 hybrid_clock.cc:648] HybridClock initialized: now 1786056131205302 us; error 27 us; skew 500 ppm
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:11.207865 20548 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:11.208395 20548 webserver.cc:460] Webserver started at http://127.19.74.194:41397/ using document root <none> and password file <none>
I20260806 22:42:11.208647 20548 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:11.208717 20548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:11.208853 20548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:11.209873 20548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "5f148882addd4b2bb173b17a7a6d1096"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "ba9e4012e7eefd77b946ec3d66913802"
server_key_iv: "6d624c30c3333ef1f35bf78ce616adac"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.210224 20548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "5f148882addd4b2bb173b17a7a6d1096"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "ba9e4012e7eefd77b946ec3d66913802"
server_key_iv: "6d624c30c3333ef1f35bf78ce616adac"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.211638 20548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:11.212370 20564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.212539 20548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:11.212663 20548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "5f148882addd4b2bb173b17a7a6d1096"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "ba9e4012e7eefd77b946ec3d66913802"
server_key_iv: "6d624c30c3333ef1f35bf78ce616adac"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.212738 20548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:11.224360 20548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:11.226078 20548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:11.226243 20548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:11.226492 20548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:11.226818 20548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:11.226852 20548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.226873 20548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:11.226918 20548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.232616 20548 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:44225
I20260806 22:42:11.232658 20681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:44225 every 8 connection(s)
I20260806 22:42:11.232947 20548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:11.236547 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 20548
I20260806 22:42:11.236636 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:11.236786 19755 external_mini_cluster.cc:1428] Setting key 90b46a38cdc4d75d936cc6174cbb1228
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:11.242813 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.233579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:34245 (local address 127.19.74.254:43959)
0806 22:42:11.233656 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.233658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.233841 (+ 183us) server_negotiation.cc:407] Connection header received
0806 22:42:11.234813 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.234814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.234825 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.234877 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:11.235417 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.235900 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.237116 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.237334 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.237428 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.239768 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:11.239770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:11.239770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:11.239778 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:11.241217 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.241739 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.241740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.241740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.241802 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.242108 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.242109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.242109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.242222 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:11.242277 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.242670 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.242751 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:42:11.243055 20684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.233545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:43959 (local address 127.19.74.194:34245)
0806 22:42:11.233729 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:11.233763 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:11.234722 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:11.234927 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:11.234930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:11.235177 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:11.235349 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.235353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.235979 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.235979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.237027 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.237029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.237462 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.237463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.237632 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.238113 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:11.238122 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:11.239665 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:11.241303 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:11.241310 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:11.241312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:11.241660 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:11.241864 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:11.241865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:11.241865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:11.242031 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:11.242292 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:11.242299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:11.242466 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:42:11.242591 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 22:42:11.242803 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":111,"threads_started":1}
I20260806 22:42:11.243672 20682 heartbeater.cc:347] Connected to a master server at 127.19.74.254:43959
I20260806 22:42:11.243782 20682 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:11.243999 20682 heartbeater.cc:511] Master 127.19.74.254:43959 requested a full tablet report, sending...
I20260806 22:42:11.244427 20344 ts_manager.cc:194] Registered new tserver with Master: 5f148882addd4b2bb173b17a7a6d1096 (127.19.74.194:44225)
I20260806 22:42:11.244987 20344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:34245
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:11.265272 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43959
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:11.345548 20687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:11.345779 20687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:11.345815 20687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:11.346930 20687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:11.346995 20687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:11.347038 20687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:11.348456 20687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43959
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:11.348712 20687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:11.348932 20687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:11.353863 20693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:11.353835 20695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:11.353827 20692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:11.354149 20687 server_base.cc:1034] running on GCE node
I20260806 22:42:11.354393 20687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:11.354691 20687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:11.355878 20687 hybrid_clock.cc:648] HybridClock initialized: now 1786056131355851 us; error 35 us; skew 500 ppm
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:11.358266 20687 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:11.358727 20687 webserver.cc:460] Webserver started at http://127.19.74.195:43113/ using document root <none> and password file <none>
I20260806 22:42:11.358911 20687 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:11.358947 20687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:11.359053 20687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:11.359982 20687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "8919611df4d946e0a6a4925ac20b6b8f"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "229fe155618707813da895e44c58a198"
server_key_iv: "fe809c8148a6c1f5abe47cb1f52dead7"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.360275 20687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "8919611df4d946e0a6a4925ac20b6b8f"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "229fe155618707813da895e44c58a198"
server_key_iv: "fe809c8148a6c1f5abe47cb1f52dead7"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.361464 20687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:42:11.362085 20702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.362244 20687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.362321 20687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "8919611df4d946e0a6a4925ac20b6b8f"
format_stamp: "Formatted at 2026-08-06 22:42:11 on dist-test-slave-pgkr"
server_key: "229fe155618707813da895e44c58a198"
server_key_iv: "fe809c8148a6c1f5abe47cb1f52dead7"
server_key_version: "encryptionkey@0"
I20260806 22:42:11.362371 20687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:11.372045 20687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:11.373718 20687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:11.373819 20687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:11.374006 20687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:11.374244 20687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:11.374274 20687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.374293 20687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:11.374306 20687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:11.379818 20687 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:43425
I20260806 22:42:11.379891 20826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:43425 every 8 connection(s)
I20260806 22:42:11.380191 20687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:11.388746 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.380850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:34497 (local address 127.19.74.254:43959)
0806 22:42:11.380890 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.380893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.381089 (+ 196us) server_negotiation.cc:407] Connection header received
0806 22:42:11.381958 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.381959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.381969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.382025 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:11.382586 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.383075 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.383996 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.384179 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.384230 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.386285 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:11.386287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:11.386287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:11.386293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:11.387684 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.388020 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.388021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.388021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.388071 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.388266 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.388266 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.388267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.388362 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:11.388410 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.388627 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.388685 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:42:11.388875 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.380816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:43959 (local address 127.19.74.195:34497)
0806 22:42:11.380971 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:11.381009 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:11.381872 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:11.382087 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:11.382090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:11.382328 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:11.382505 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.382508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.383154 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.383155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.383908 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.383909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.384251 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.384252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.384352 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.384848 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:11.384856 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:11.386199 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:11.387751 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:11.387756 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:11.387758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:11.387956 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:11.388125 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:11.388125 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:11.388125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:11.388213 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:11.388413 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:11.388417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:11.388513 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 22:42:11.388588 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:42:11.388702 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 22:42:11.389307 20827 heartbeater.cc:347] Connected to a master server at 127.19.74.254:43959
I20260806 22:42:11.389411 20827 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:11.389577 20827 heartbeater.cc:511] Master 127.19.74.254:43959 requested a full tablet report, sending...
I20260806 22:42:11.389932 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 20687
I20260806 22:42:11.389940 20344 ts_manager.cc:194] Registered new tserver with Master: 8919611df4d946e0a6a4925ac20b6b8f (127.19.74.195:43425)
I20260806 22:42:11.390053 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:11.390213 19755 external_mini_cluster.cc:1428] Setting key 08b5cb7f4bad2dab1782bfce66728bb2
I20260806 22:42:11.390367 20344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:34497
I20260806 22:42:11.391239 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:11.398191 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.392936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.19.74.254:43959)
0806 22:42:11.392970 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.392971 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.393103 (+ 132us) server_negotiation.cc:407] Connection header received
0806 22:42:11.393125 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.393126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.393133 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.393184 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:11.393681 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.394105 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.394834 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.395059 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.395131 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.396239 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:11.396239 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:11.396240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:11.396244 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:11.397327 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.397611 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.397611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.397612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.397657 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.397767 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.397767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.397767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.397872 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:11.397939 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.398024 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.398077 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:42:11.400182 20344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:11.401684 20344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:11.407882 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.404045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.19.74.195:43425)
0806 22:42:11.404189 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.404193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.404202 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:11.404242 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.404244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.404255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.404342 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:11.404476 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.405185 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.406842 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.407475 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.407538 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.407590 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.407765 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.407806 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260806 22:42:11.407912 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.403693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43425 (local address 127.0.0.1:36776)
0806 22:42:11.403839 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:11.403876 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:11.403940 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:11.404342 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:11.404343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:11.404346 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:11.404394 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.404397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.405288 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.405289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.406463 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.406466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.407554 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.407555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.407644 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.407657 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:42:11.407739 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:42:11.407806 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":47,"threads_started":1}
I20260806 22:42:11.407917 20835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.403717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:41743 (local address 127.0.0.1:38222)
0806 22:42:11.404056 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:11.404065 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:11.404105 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:11.404298 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:11.404300 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:11.404304 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:11.404361 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.404362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.405621 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.405622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.406651 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.406654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.407645 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.407646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.407693 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.407696 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:11.407766 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:42:11.407813 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":2,"thread_start_us":38,"threads_started":1}
I20260806 22:42:11.408046 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.403703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:44225 (local address 127.0.0.1:35854)
0806 22:42:11.403918 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:11.403929 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:11.403985 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:11.404635 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:11.404636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:11.404638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:11.404678 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.404679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.405863 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.405864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.407065 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.407068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.407813 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.407814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.407873 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.407877 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:11.407917 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 22:42:11.407954 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":52,"threads_started":1}
I20260806 22:42:11.408391 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.403850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.19.74.194:44225)
0806 22:42:11.404467 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.404471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.404481 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:11.404521 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.404522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.404536 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.404747 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:11.404819 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.405758 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.407171 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.407723 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.408196 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.408261 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.408288 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.408325 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":98,"threads_started":1}
I20260806 22:42:11.408524 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.403917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.19.74.193:41743)
0806 22:42:11.404127 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.404132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.404142 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:11.404182 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.404184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.404198 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.404294 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:11.404449 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.405507 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.406738 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.407458 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.407536 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.407591 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.408379 (+ 788us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.408415 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":82,"threads_started":1}
I20260806 22:42:11.408684 20756 tablet_service.cc:1467] Processing CreateTablet for tablet 51433e83a38744229bb4943d661b54ce (DEFAULT_TABLE table=test-table [id=c5cef305742a48b0a06f2a80af43cd99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:11.409034 20756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51433e83a38744229bb4943d661b54ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:11.408980 20613 tablet_service.cc:1467] Processing CreateTablet for tablet 51433e83a38744229bb4943d661b54ce (DEFAULT_TABLE table=test-table [id=c5cef305742a48b0a06f2a80af43cd99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:11.409287 20613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51433e83a38744229bb4943d661b54ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:11.410996 20472 tablet_service.cc:1467] Processing CreateTablet for tablet 51433e83a38744229bb4943d661b54ce (DEFAULT_TABLE table=test-table [id=c5cef305742a48b0a06f2a80af43cd99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:11.411311 20472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51433e83a38744229bb4943d661b54ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:11.412822 20844 tablet_bootstrap.cc:492] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f: Bootstrap starting.
I20260806 22:42:11.413537 20844 tablet_bootstrap.cc:654] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:11.414143 20844 log.cc:824] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:11.414469 20843 tablet_bootstrap.cc:492] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096: Bootstrap starting.
I20260806 22:42:11.414950 20846 tablet_bootstrap.cc:492] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55: Bootstrap starting.
I20260806 22:42:11.414999 20844 tablet_bootstrap.cc:492] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f: No bootstrap required, opened a new log
I20260806 22:42:11.415069 20844 ts_tablet_manager.cc:1397] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:11.415233 20843 tablet_bootstrap.cc:654] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:11.415688 20846 tablet_bootstrap.cc:654] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:11.415817 20843 log.cc:824] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:11.416234 20846 log.cc:824] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:11.416658 20843 tablet_bootstrap.cc:492] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096: No bootstrap required, opened a new log
I20260806 22:42:11.416734 20843 ts_tablet_manager.cc:1397] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 22:42:11.417032 20846 tablet_bootstrap.cc:492] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55: No bootstrap required, opened a new log
I20260806 22:42:11.417107 20846 ts_tablet_manager.cc:1397] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:11.417037 20844 raft_consensus.cc:348] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.417228 20844 raft_consensus.cc:374] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:11.417270 20844 raft_consensus.cc:729] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8919611df4d946e0a6a4925ac20b6b8f, State: Initialized, Role: FOLLOWER
I20260806 22:42:11.417379 20844 consensus_queue.cc:260] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.417650 20844 ts_tablet_manager.cc:1428] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:42:11.417765 20827 heartbeater.cc:503] Master 127.19.74.254:43959 was elected leader, sending a full tablet report...
I20260806 22:42:11.418672 20843 raft_consensus.cc:348] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.418829 20843 raft_consensus.cc:374] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:11.418874 20843 raft_consensus.cc:729] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f148882addd4b2bb173b17a7a6d1096, State: Initialized, Role: FOLLOWER
I20260806 22:42:11.418980 20843 consensus_queue.cc:260] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.418998 20846 raft_consensus.cc:348] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.419173 20846 raft_consensus.cc:374] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:11.419220 20846 raft_consensus.cc:729] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571db821521d434eacaa666bafa02f55, State: Initialized, Role: FOLLOWER
I20260806 22:42:11.419253 20843 ts_tablet_manager.cc:1428] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 22:42:11.419250 20682 heartbeater.cc:503] Master 127.19.74.254:43959 was elected leader, sending a full tablet report...
I20260806 22:42:11.419348 20846 consensus_queue.cc:260] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.419649 20846 ts_tablet_manager.cc:1428] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:42:11.419863 20543 heartbeater.cc:503] Master 127.19.74.254:43959 was elected leader, sending a full tablet report...
W20260806 22:42:11.483466 20683 tablet.cc:2369] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:11.585777 20544 tablet.cc:2369] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:11.630770 20828 tablet.cc:2369] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:11.666801 20850 raft_consensus.cc:482] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:11.666954 20850 raft_consensus.cc:504] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.667426 20850 leader_election.cc:288] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8919611df4d946e0a6a4925ac20b6b8f (127.19.74.195:43425), 571db821521d434eacaa666bafa02f55 (127.19.74.193:41743)
I20260806 22:42:11.670192 20684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.667615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:41743 (local address 127.19.74.194:37503)
0806 22:42:11.667675 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:11.667687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:11.667741 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:11.667928 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:11.667929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:11.667931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:11.668000 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.668001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.668611 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.668612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.669377 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.669379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.669929 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.669930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.670006 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.670009 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:11.670077 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:42:11.670122 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:42:11.670192 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.667695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:37503 (local address 127.19.74.193:41743)
0806 22:42:11.667748 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.667750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.667765 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:42:11.667801 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.667802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.667814 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.667904 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:11.668072 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.668528 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.669482 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.669845 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.669931 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.669940 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.670083 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.670121 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:42:11.670457 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.667743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:41689 (local address 127.19.74.195:43425)
0806 22:42:11.667805 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.667807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.667956 (+ 149us) server_negotiation.cc:407] Connection header received
0806 22:42:11.668010 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.668011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.668020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.668091 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:11.668267 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.668719 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.669680 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.670059 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.670149 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.670157 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.670340 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.670388 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:42:11.670545 20494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51433e83a38744229bb4943d661b54ce" candidate_uuid: "5f148882addd4b2bb173b17a7a6d1096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571db821521d434eacaa666bafa02f55" is_pre_election: true
I20260806 22:42:11.670701 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.667637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43425 (local address 127.19.74.194:41689)
0806 22:42:11.667865 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:11.667879 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:11.667955 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:11.668101 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:11.668102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:11.668103 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:11.668189 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.668191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.668804 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.668805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.669581 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:11.669584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.670149 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.670150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:11.670225 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.670228 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:11.670553 (+ 325us) client_negotiation.cc:240] Negotiation successful
0806 22:42:11.670607 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":59,"thread_start_us":121,"threads_started":1}
I20260806 22:42:11.670766 20494 raft_consensus.cc:2454] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f148882addd4b2bb173b17a7a6d1096 in term 0.
I20260806 22:42:11.671056 20566 leader_election.cc:302] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 571db821521d434eacaa666bafa02f55, 5f148882addd4b2bb173b17a7a6d1096; no voters:
I20260806 22:42:11.671065 20777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51433e83a38744229bb4943d661b54ce" candidate_uuid: "5f148882addd4b2bb173b17a7a6d1096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8919611df4d946e0a6a4925ac20b6b8f" is_pre_election: true
I20260806 22:42:11.671207 20850 raft_consensus.cc:2781] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:11.671252 20777 raft_consensus.cc:2454] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f148882addd4b2bb173b17a7a6d1096 in term 0.
I20260806 22:42:11.671265 20850 raft_consensus.cc:482] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:11.671301 20850 raft_consensus.cc:3037] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:11.672053 20850 raft_consensus.cc:504] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.672209 20850 leader_election.cc:288] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [CANDIDATE]: Term 1 election: Requested vote from peers 8919611df4d946e0a6a4925ac20b6b8f (127.19.74.195:43425), 571db821521d434eacaa666bafa02f55 (127.19.74.193:41743)
I20260806 22:42:11.672377 20777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51433e83a38744229bb4943d661b54ce" candidate_uuid: "5f148882addd4b2bb173b17a7a6d1096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8919611df4d946e0a6a4925ac20b6b8f"
I20260806 22:42:11.672405 20494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51433e83a38744229bb4943d661b54ce" candidate_uuid: "5f148882addd4b2bb173b17a7a6d1096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571db821521d434eacaa666bafa02f55"
I20260806 22:42:11.672463 20777 raft_consensus.cc:3037] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:11.672472 20494 raft_consensus.cc:3037] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:11.673321 20777 raft_consensus.cc:2454] T 51433e83a38744229bb4943d661b54ce P 8919611df4d946e0a6a4925ac20b6b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f148882addd4b2bb173b17a7a6d1096 in term 1.
I20260806 22:42:11.673321 20494 raft_consensus.cc:2454] T 51433e83a38744229bb4943d661b54ce P 571db821521d434eacaa666bafa02f55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f148882addd4b2bb173b17a7a6d1096 in term 1.
I20260806 22:42:11.673514 20566 leader_election.cc:302] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 571db821521d434eacaa666bafa02f55, 5f148882addd4b2bb173b17a7a6d1096; no voters:
I20260806 22:42:11.673646 20850 raft_consensus.cc:2781] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:11.673830 20850 raft_consensus.cc:686] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [term 1 LEADER]: Becoming Leader. State: Replica: 5f148882addd4b2bb173b17a7a6d1096, State: Running, Role: LEADER
I20260806 22:42:11.673942 20850 consensus_queue.cc:237] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } }
I20260806 22:42:11.674855 20343 catalog_manager.cc:5495] T 51433e83a38744229bb4943d661b54ce P 5f148882addd4b2bb173b17a7a6d1096 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f148882addd4b2bb173b17a7a6d1096 (127.19.74.194). New cstate: current_term: 1 leader_uuid: "5f148882addd4b2bb173b17a7a6d1096" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8919611df4d946e0a6a4925ac20b6b8f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f148882addd4b2bb173b17a7a6d1096" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "571db821521d434eacaa666bafa02f55" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41743 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:11.694710 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.687055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.19.74.193:41743)
0806 22:42:11.687129 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.687131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.687229 (+ 98us) server_negotiation.cc:407] Connection header received
0806 22:42:11.687325 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.687326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.687335 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.687433 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:11.688008 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.688475 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.689200 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.689408 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.689485 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.692215 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:11.692224 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:11.692225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:11.692241 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:11.693684 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.693997 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.694000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.694001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.694061 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.694227 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.694228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.694228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.694357 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:11.694457 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.694514 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.694573 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:11.718014 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.710997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.19.74.193:41743)
0806 22:42:11.711074 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.711077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.711182 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:42:11.711291 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.711292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.711303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.711373 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:11.711938 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.712417 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.713144 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.713333 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.713408 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.715526 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:11.715527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:11.715527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:11.715535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:11.716861 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.717213 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.717214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.717214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.717268 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.717466 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.717467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.717467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.717566 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:11.717665 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.717838 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.717915 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:11.739697 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:11.733047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.19.74.193:41743)
0806 22:42:11.733117 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:11.733120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:11.733204 (+ 84us) server_negotiation.cc:407] Connection header received
0806 22:42:11.733297 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:11.733298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:11.733308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:11.733381 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:11.733964 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.734451 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.735247 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:11.735446 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:11.735548 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:11.737661 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:11.737662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:11.737663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:11.737669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:11.738837 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.739127 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.739128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.739129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.739174 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:11.739330 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:11.739330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:11.739331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:11.739411 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:11.739479 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:11.739574 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:42:11.739636 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 22:42:11.740175 20451 tablet_service.cc:2908] Rejecting scan request for tablet 51433e83a38744229bb4943d661b54ce: Uninitialized: clean time has not yet been initialized
I20260806 22:42:11.741338 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 20407
I20260806 22:42:11.747663 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 20548
I20260806 22:42:11.753925 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 20687
I20260806 22:42:11.759268 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 20310
2026-08-06T22:42:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2217 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20867](info): set up 2 sockets
Aug 06 22:42:11 dist-test-slave-pgkr krb5kdc[20867](info): commencing operation
krb5kdc: starting...
W20260806 22:42:12.812860 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:12 dist-test-slave-pgkr krb5kdc[20867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:12.931317 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:36997
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:36997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:13.017385 20880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:13.017634 20880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:13.017668 20880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:13.019027 20880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:13.019068 20880 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:42:13.019080 20880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:13.019100 20880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:13.019112 20880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:13.019124 20880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:13.020624 20880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:36997
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:36997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:13.020812 20880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:13.020990 20880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:13.025895 20888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:13.025872 20886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:13.025864 20885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:13.026178 20880 server_base.cc:1034] running on GCE node
I20260806 22:42:13.026430 20880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:13.026770 20880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:13.027944 20880 hybrid_clock.cc:648] HybridClock initialized: now 1786056133027928 us; error 28 us; skew 500 ppm
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:13.034014 20880 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:13.034518 20880 webserver.cc:460] Webserver started at http://127.19.74.254:32975/ using document root <none> and password file <none>
I20260806 22:42:13.034791 20880 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:13.034862 20880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:13.035015 20880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:13.036132 20880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "44962028c32b438985d05c7c89d534c6"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "f27c01ed03b828341c1ae83c5fda9ac6"
server_key_iv: "511a844a0488a907232355274a0f81cb"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.036500 20880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "44962028c32b438985d05c7c89d534c6"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "f27c01ed03b828341c1ae83c5fda9ac6"
server_key_iv: "511a844a0488a907232355274a0f81cb"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.037866 20880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:13.038690 20895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:13.038870 20880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:13.038977 20880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "44962028c32b438985d05c7c89d534c6"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "f27c01ed03b828341c1ae83c5fda9ac6"
server_key_iv: "511a844a0488a907232355274a0f81cb"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.039062 20880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:13.049280 20880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:13.051687 20880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:13.051853 20880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:13.057018 20880 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:36997
I20260806 22:42:13.057082 20960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:36997 every 8 connection(s)
I20260806 22:42:13.057405 20880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:13.058043 20961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.060372 20961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6: Bootstrap starting.
I20260806 22:42:13.061009 20961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.061318 20961 log.cc:824] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:13.062021 20961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6: No bootstrap required, opened a new log
I20260806 22:42:13.064304 20961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44962028c32b438985d05c7c89d534c6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 36997 } }
I20260806 22:42:13.064468 20961 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.064518 20961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44962028c32b438985d05c7c89d534c6, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.064666 20961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44962028c32b438985d05c7c89d534c6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 36997 } }
I20260806 22:42:13.064752 20961 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:13.064795 20961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:13.064847 20961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.065388 20961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44962028c32b438985d05c7c89d534c6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 36997 } }
I20260806 22:42:13.065508 20961 leader_election.cc:302] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44962028c32b438985d05c7c89d534c6; no voters:
I20260806 22:42:13.065698 20961 leader_election.cc:288] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:13.065743 20964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:13.065929 20964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [term 1 LEADER]: Becoming Leader. State: Replica: 44962028c32b438985d05c7c89d534c6, State: Running, Role: LEADER
I20260806 22:42:13.065973 20961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:13.066105 20964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44962028c32b438985d05c7c89d534c6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 36997 } }
I20260806 22:42:13.066430 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 20880
I20260806 22:42:13.066532 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:13.066737 19755 external_mini_cluster.cc:1428] Setting key d8562bc72992021e3630c21675f0b0ec
I20260806 22:42:13.067278 20965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44962028c32b438985d05c7c89d534c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44962028c32b438985d05c7c89d534c6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 36997 } } }
I20260806 22:42:13.067422 20965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:13.067585 20966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44962028c32b438985d05c7c89d534c6. Latest consensus state: current_term: 1 leader_uuid: "44962028c32b438985d05c7c89d534c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44962028c32b438985d05c7c89d534c6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 36997 } } }
I20260806 22:42:13.067979 20966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6 [sys.catalog]: This master's current role is: LEADER
W20260806 22:42:13.068034 20980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:13.068135 20980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:13.068573 20984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:13.068758 20984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:13.070317 20984 catalog_manager.cc:1347] Generated new cluster ID: e3000151fb5245e5a204a6f389537667
I20260806 22:42:13.070370 20984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:13.078902 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.067805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.19.74.254:36997)
0806 22:42:13.068151 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.068161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.068200 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:42:13.068310 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.068316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.068357 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.068466 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.069259 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.070537 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.071659 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.072266 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.072358 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.075214 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.075219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.075219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.075231 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.077179 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.077564 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.077567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.077567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.077689 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.077941 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.077942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.077942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.078276 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.078440 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.078510 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.078678 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":73,"threads_started":1}
I20260806 22:42:13.094882 20984 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:13.095386 20984 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:13.101881 20984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6: Generated new TSK 0
I20260806 22:42:13.102088 20984 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:13.133324 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:36997
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:13.215905 20988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:13.216125 20988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:13.216163 20988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:13.217279 20988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:13.217356 20988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:13.217392 20988 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:13.217440 20988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:13.218964 20988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:36997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:13.219242 20988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:13.219507 20988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:42:13.224660 20988 server_base.cc:1034] running on GCE node
W20260806 22:42:13.224633 20995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:13.224642 20994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:13.224642 20997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:13.225108 20988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:13.225322 20988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:13.226460 20988 hybrid_clock.cc:648] HybridClock initialized: now 1786056133226449 us; error 27 us; skew 500 ppm
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:13.229090 20988 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:13.229560 20988 webserver.cc:460] Webserver started at http://127.19.74.193:43985/ using document root <none> and password file <none>
I20260806 22:42:13.229784 20988 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:13.229828 20988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:13.229993 20988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:13.231088 20988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "02660991fa2c43d3ab2b7c758edacf0b"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "6786349b6a8b51edceab366b3f82882b"
server_key_iv: "475d4a07ddc99688ab3482e3c47959f8"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.231446 20988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "02660991fa2c43d3ab2b7c758edacf0b"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "6786349b6a8b51edceab366b3f82882b"
server_key_iv: "475d4a07ddc99688ab3482e3c47959f8"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.232762 20988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:13.233441 21004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:13.233630 20988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:13.233736 20988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "02660991fa2c43d3ab2b7c758edacf0b"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "6786349b6a8b51edceab366b3f82882b"
server_key_iv: "475d4a07ddc99688ab3482e3c47959f8"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.233806 20988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:13.259481 20988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:13.261881 20988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:13.262048 20988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:13.262790 20988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:13.262843 20988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:13.262874 20988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:13.262895 20988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:13.270217 20988 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:34723
I20260806 22:42:13.270658 20988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:13.271052 21122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:34723 every 8 connection(s)
I20260806 22:42:13.272907 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.263165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:48363 (local address 127.19.74.254:36997)
0806 22:42:13.263230 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.263232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.263471 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:42:13.263997 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.263999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.264018 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.264069 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.264764 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.265306 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.266517 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.266841 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.266927 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.270242 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.270244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.270244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.270252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.271775 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.272141 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.272142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.272142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.272196 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.272390 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.272391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.272391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.272490 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.272543 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.272782 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.272845 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:42:13.273058 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.263180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:36997 (local address 127.19.74.193:48363)
0806 22:42:13.263346 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.263387 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.263881 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.264163 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.264167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.264458 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:13.264661 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.264666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.265437 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.265438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.266428 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.266431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.266940 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.266941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.267111 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.267623 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:13.267633 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:13.270146 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:13.271850 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.271854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.271856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.272068 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.272254 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.272254 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.272254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.272340 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.272549 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:13.272555 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:13.272671 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.272753 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.272870 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260806 22:42:13.273623 21123 heartbeater.cc:347] Connected to a master server at 127.19.74.254:36997
I20260806 22:42:13.273732 21123 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:13.273931 21123 heartbeater.cc:511] Master 127.19.74.254:36997 requested a full tablet report, sending...
I20260806 22:42:13.274508 20913 ts_manager.cc:194] Registered new tserver with Master: 02660991fa2c43d3ab2b7c758edacf0b (127.19.74.193:34723)
I20260806 22:42:13.275405 20913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:48363
I20260806 22:42:13.279176 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 20988
I20260806 22:42:13.279294 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:13.279500 19755 external_mini_cluster.cc:1428] Setting key 4dac1eb140a17bc7e4811c4115a8a201
I20260806 22:42:13.280131 21129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.274795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:36997 (local address 127.19.74.193:49045)
0806 22:42:13.274944 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.274956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.275018 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.275133 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.275134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.275323 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:13.275382 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.275384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.276097 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.276098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.276656 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.276659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.276960 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.276961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.277015 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.277455 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:13.277460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:13.277739 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:13.279051 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.279052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.279053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.279284 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.279504 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.279505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.279505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.279576 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.279844 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:13.279845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:13.279892 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.279959 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.280061 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260806 22:42:13.280201 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.274834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:49045 (local address 127.19.74.254:36997)
0806 22:42:13.274875 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.274877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.275009 (+ 132us) server_negotiation.cc:407] Connection header received
0806 22:42:13.275072 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.275072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.275081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.275127 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.275455 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.276010 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.276729 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.276898 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.276946 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.277809 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.277810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.277810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.277815 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.278974 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.279369 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.279371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.279371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.279435 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.279647 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.279648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.279648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.279788 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.279848 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.280005 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.280104 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:13.304628 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:36997
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:13.385833 21132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:13.386032 21132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:13.386055 21132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:13.387159 21132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:13.387194 21132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:13.387233 21132 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:13.387249 21132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:13.388650 21132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:36997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:13.388906 21132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:13.389096 21132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:13.394174 21140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:13.394263 21137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:13.394341 21138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:13.394353 21132 server_base.cc:1034] running on GCE node
I20260806 22:42:13.394682 21132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:13.394971 21132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:13.396124 21132 hybrid_clock.cc:648] HybridClock initialized: now 1786056133396105 us; error 38 us; skew 500 ppm
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:13.398696 21132 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:13.399163 21132 webserver.cc:460] Webserver started at http://127.19.74.194:43481/ using document root <none> and password file <none>
I20260806 22:42:13.399358 21132 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:13.399397 21132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:13.399488 21132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:13.400549 21132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "97d24554bfdd450098cd2490fab0fc8e"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "6d3f0c786b442e25956ec2b1cef82ea9"
server_key_iv: "a2ba9157687941bc5356a7e02a241175"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.400862 21132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "97d24554bfdd450098cd2490fab0fc8e"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "6d3f0c786b442e25956ec2b1cef82ea9"
server_key_iv: "a2ba9157687941bc5356a7e02a241175"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.402158 21132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:13.402963 21148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:13.403159 21132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:13.403250 21132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "97d24554bfdd450098cd2490fab0fc8e"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "6d3f0c786b442e25956ec2b1cef82ea9"
server_key_iv: "a2ba9157687941bc5356a7e02a241175"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.403323 21132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:13.412367 21132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:13.414227 21132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:13.414340 21132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:13.414875 21132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:13.414915 21132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:13.414937 21132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:13.414949 21132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:13.421922 21132 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:45549
I20260806 22:42:13.422015 21265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:45549 every 8 connection(s)
I20260806 22:42:13.422302 21132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:13.424660 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.415351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:36671 (local address 127.19.74.254:36997)
0806 22:42:13.415444 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.415447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.415612 (+ 165us) server_negotiation.cc:407] Connection header received
0806 22:42:13.416089 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.416091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.416104 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.416196 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.417054 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.417559 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.419146 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.419450 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.419522 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.422098 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.422099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.422100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.422108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.423603 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.423948 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.423949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.423949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.423995 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.424164 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.424165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.424165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.424256 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.424301 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.424528 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.424587 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 22:42:13.424853 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.415303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:36997 (local address 127.19.74.194:36671)
0806 22:42:13.415469 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.415519 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.415968 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.416242 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.416246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.416640 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:13.416920 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.416926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.417669 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.417670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.419054 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.419057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.419555 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.419556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.419702 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.420258 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:13.420267 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:13.421986 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:13.423681 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.423686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.423688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.423884 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.424033 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.424034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.424034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.424113 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.424307 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:13.424312 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:13.424416 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.424496 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.424626 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":103,"threads_started":1}
I20260806 22:42:13.425333 21266 heartbeater.cc:347] Connected to a master server at 127.19.74.254:36997
I20260806 22:42:13.425441 21266 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:13.425613 21266 heartbeater.cc:511] Master 127.19.74.254:36997 requested a full tablet report, sending...
I20260806 22:42:13.425995 20912 ts_manager.cc:194] Registered new tserver with Master: 97d24554bfdd450098cd2490fab0fc8e (127.19.74.194:45549)
I20260806 22:42:13.426501 20912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:36671
I20260806 22:42:13.429626 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 21132
I20260806 22:42:13.429703 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:13.429924 19755 external_mini_cluster.cc:1428] Setting key 47152652416e040fbf44e89be4d20483
I20260806 22:42:13.431977 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.426368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:42605 (local address 127.19.74.254:36997)
0806 22:42:13.426422 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.426424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.426507 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:42:13.426536 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.426537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.426546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.426606 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.426992 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.427430 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.428088 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.428284 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.428365 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.429249 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.429250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.429251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.429257 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.430796 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.431161 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.431162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.431162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.431216 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.431421 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.431422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.431423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.431569 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.431656 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.431818 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.431890 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:42:13.431993 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.426280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:36997 (local address 127.19.74.194:42605)
0806 22:42:13.426437 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.426446 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.426501 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.426694 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.426695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.426860 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:13.426912 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.426913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.427523 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.427524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.428006 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.428007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.428365 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.428366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.428443 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.428850 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:13.428855 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:13.429144 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:13.430892 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.430893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.430893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.431064 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.431290 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.431291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.431291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.431348 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.431647 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:13.431648 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:13.431693 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.431792 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.431892 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:13.454780 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:36997
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:13.536808 21275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:13.537042 21275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:13.537065 21275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:13.538245 21275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:13.538283 21275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:13.538303 21275 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:13.538321 21275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:13.539795 21275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:36997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:13.539986 21275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:13.540162 21275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:13.545137 21283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:13.545145 21280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:13.545202 21281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:13.545603 21275 server_base.cc:1034] running on GCE node
I20260806 22:42:13.545809 21275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:13.546048 21275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:13.547214 21275 hybrid_clock.cc:648] HybridClock initialized: now 1786056133547198 us; error 30 us; skew 500 ppm
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:13.549775 21275 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:13.550225 21275 webserver.cc:460] Webserver started at http://127.19.74.195:37375/ using document root <none> and password file <none>
I20260806 22:42:13.550485 21275 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:13.550561 21275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:13.550737 21275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:13.551759 21275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "0342d83e700845ce8506ca61be313acd"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "ebf56b52376c99244cfa80d643bdd732"
server_key_iv: "4db499710b6953818850d8569b6cb2ad"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.552136 21275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "0342d83e700845ce8506ca61be313acd"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "ebf56b52376c99244cfa80d643bdd732"
server_key_iv: "4db499710b6953818850d8569b6cb2ad"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.553653 21275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:13.554373 21290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:13.554554 21275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:13.554688 21275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "0342d83e700845ce8506ca61be313acd"
format_stamp: "Formatted at 2026-08-06 22:42:13 on dist-test-slave-pgkr"
server_key: "ebf56b52376c99244cfa80d643bdd732"
server_key_iv: "4db499710b6953818850d8569b6cb2ad"
server_key_version: "encryptionkey@0"
I20260806 22:42:13.554768 21275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:13.566526 21275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:13.568717 21275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:13.568895 21275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:13.569495 21275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:13.569648 21275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:13.569705 21275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:13.569751 21275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:13.577013 21275 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:32785
I20260806 22:42:13.577066 21407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:32785 every 8 connection(s)
I20260806 22:42:13.577473 21275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:13.579424 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 21275
I20260806 22:42:13.579506 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:13.579665 19755 external_mini_cluster.cc:1428] Setting key c1df41781d46b30e66d0aafc6997fd18
I20260806 22:42:13.579916 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.569841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:39525 (local address 127.19.74.254:36997)
0806 22:42:13.569906 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.569908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.570208 (+ 300us) server_negotiation.cc:407] Connection header received
0806 22:42:13.570716 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.570718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.570730 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.570784 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.571575 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.572434 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.573942 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.574284 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.574412 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.576818 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.576819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.576819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.576827 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.578422 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.578902 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.578904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.578904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.578985 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.579249 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.579249 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.579250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.579358 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.579410 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.579729 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.579813 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:42:13.580061 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.569843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:36997 (local address 127.19.74.195:39525)
0806 22:42:13.570091 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.570128 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.570587 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.570859 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.570862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.571225 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:13.571488 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.571494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.572530 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.572531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.573842 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.573845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.574435 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.574440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.574601 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.575146 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:13.575171 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:13.576709 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:13.578533 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.578538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.578539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.578826 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.579064 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.579064 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.579064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.579166 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.579423 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:13.579429 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:13.579580 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.579681 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.579833 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":90,"threads_started":1}
I20260806 22:42:13.580564 21408 heartbeater.cc:347] Connected to a master server at 127.19.74.254:36997
I20260806 22:42:13.580678 21408 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:13.580919 21408 heartbeater.cc:511] Master 127.19.74.254:36997 requested a full tablet report, sending...
I20260806 22:42:13.581362 20912 ts_manager.cc:194] Registered new tserver with Master: 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785)
I20260806 22:42:13.581796 20912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:39525
I20260806 22:42:13.586735 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.581740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:36997 (local address 127.19.74.195:57433)
0806 22:42:13.581866 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.581877 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.581938 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.582097 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.582098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.582282 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:13.582333 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.582334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.583039 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.583039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.583498 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.583499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.583801 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.583802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.583856 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.584275 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:13.584280 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:13.584546 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:13.585859 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.585860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.585860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.586002 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.586206 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.586207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.586207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.586257 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.586465 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:13.586465 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:13.586497 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.586546 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.586669 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260806 22:42:13.586818 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.581823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:57433 (local address 127.19.74.254:36997)
0806 22:42:13.581860 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.581862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.581941 (+ 79us) server_negotiation.cc:407] Connection header received
0806 22:42:13.581998 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.581999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.582011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.582084 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.582416 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.582953 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.583570 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.583735 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.583790 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.584620 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.584621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.584621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.584626 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.585775 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.586095 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.586096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.586096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.586147 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.586314 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.586314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.586315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.586406 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.586464 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.586574 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.586697 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:13.591632 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:13.615247 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.608279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46462 (local address 127.19.74.254:36997)
0806 22:42:13.608346 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.608348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.608356 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:13.608379 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.608380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.608402 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.608477 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.608999 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.609477 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.610190 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.610382 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.610449 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.612708 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.612710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.612710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.612721 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.613922 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.614341 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.614342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.614342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.614405 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.614685 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.614686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.614687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.614794 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.614905 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.615053 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.615136 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:42:13.616780 20912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:13.618364 20912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:13.625017 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.620739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:32785 (local address 127.0.0.1:59286)
0806 22:42:13.621038 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.621048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.621110 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.621803 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.621803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.621805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:13.621840 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.621841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.623049 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.623049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.624002 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.624005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.624687 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.624689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.624768 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.624789 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.624880 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.624934 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":20,"thread_start_us":75,"threads_started":1}
I20260806 22:42:13.625030 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.620851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.19.74.195:32785)
0806 22:42:13.621646 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.621650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.621660 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:13.621705 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.621707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.621725 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.621806 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:13.621902 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.622955 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.624095 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.624596 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.624686 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.624747 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.624901 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.624942 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":91,"threads_started":1}
I20260806 22:42:13.625257 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.620739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:45549 (local address 127.0.0.1:39762)
0806 22:42:13.620988 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.621034 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.621110 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.621530 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.621533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.621536 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:13.621614 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.621616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.622927 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.622928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.624034 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.624036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.625015 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.625017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.625090 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.625095 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.625132 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.625178 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":26,"thread_start_us":67,"threads_started":1}
I20260806 22:42:13.625638 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.620960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.19.74.194:45549)
0806 22:42:13.621317 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.621322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.621333 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:13.621375 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.621377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.621394 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.621511 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:13.621684 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.622839 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.624151 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.624896 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.625347 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.625409 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.625438 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.625481 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":79,"threads_started":1}
I20260806 22:42:13.625676 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.621197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37202 (local address 127.19.74.193:34723)
0806 22:42:13.621391 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.621396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.621408 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:42:13.621453 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.621455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.621473 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.621574 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:13.621772 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.622520 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.624151 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.624905 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.624978 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.625458 (+ 480us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.625491 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.625562 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260806 22:42:13.625419 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.620948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:34723 (local address 127.0.0.1:37202)
0806 22:42:13.621133 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.621143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.621198 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.621648 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.621649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.621652 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:13.621699 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.621700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.622690 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.622691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.624028 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.624031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.625108 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.625108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.625254 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.625258 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.625322 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.625360 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
I20260806 22:42:13.626123 21342 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdbfdba15c449dc939080105f40e909 (DEFAULT_TABLE table=test-table [id=8d498db60cd94279a0b9ca1f6584f9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:13.626533 21342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdbfdba15c449dc939080105f40e909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.626426 21055 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdbfdba15c449dc939080105f40e909 (DEFAULT_TABLE table=test-table [id=8d498db60cd94279a0b9ca1f6584f9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:13.626425 21199 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdbfdba15c449dc939080105f40e909 (DEFAULT_TABLE table=test-table [id=8d498db60cd94279a0b9ca1f6584f9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:13.626811 21199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdbfdba15c449dc939080105f40e909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.626811 21055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdbfdba15c449dc939080105f40e909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.629244 21431 tablet_bootstrap.cc:492] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: Bootstrap starting.
I20260806 22:42:13.630172 21431 tablet_bootstrap.cc:654] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.630522 21432 tablet_bootstrap.cc:492] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Bootstrap starting.
I20260806 22:42:13.630543 21431 log.cc:824] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:13.630782 21433 tablet_bootstrap.cc:492] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Bootstrap starting.
I20260806 22:42:13.631421 21431 tablet_bootstrap.cc:492] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: No bootstrap required, opened a new log
I20260806 22:42:13.631457 21433 tablet_bootstrap.cc:654] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.631502 21431 ts_tablet_manager.cc:1397] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:13.631793 21433 log.cc:824] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:13.632604 21432 tablet_bootstrap.cc:654] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.632637 21433 tablet_bootstrap.cc:492] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: No bootstrap required, opened a new log
I20260806 22:42:13.632719 21433 ts_tablet_manager.cc:1397] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:13.632944 21432 log.cc:824] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:13.633620 21431 raft_consensus.cc:348] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.633792 21432 tablet_bootstrap.cc:492] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: No bootstrap required, opened a new log
I20260806 22:42:13.633783 21431 raft_consensus.cc:374] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.633826 21431 raft_consensus.cc:729] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0342d83e700845ce8506ca61be313acd, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.633845 21432 ts_tablet_manager.cc:1397] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:42:13.633989 21431 consensus_queue.cc:260] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.634296 21431 ts_tablet_manager.cc:1428] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:42:13.634357 21408 heartbeater.cc:503] Master 127.19.74.254:36997 was elected leader, sending a full tablet report...
I20260806 22:42:13.634760 21433 raft_consensus.cc:348] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.634984 21433 raft_consensus.cc:374] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.635071 21433 raft_consensus.cc:729] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d24554bfdd450098cd2490fab0fc8e, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.635262 21433 consensus_queue.cc:260] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.635618 21433 ts_tablet_manager.cc:1428] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:42:13.635632 21266 heartbeater.cc:503] Master 127.19.74.254:36997 was elected leader, sending a full tablet report...
I20260806 22:42:13.635914 21432 raft_consensus.cc:348] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.636078 21432 raft_consensus.cc:374] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.636118 21432 raft_consensus.cc:729] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02660991fa2c43d3ab2b7c758edacf0b, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.636253 21432 consensus_queue.cc:260] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.636575 21432 ts_tablet_manager.cc:1428] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:42:13.636620 21123 heartbeater.cc:503] Master 127.19.74.254:36997 was elected leader, sending a full tablet report...
W20260806 22:42:13.673084 21267 tablet.cc:2369] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:13.743788 21438 raft_consensus.cc:482] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:13.743939 21438 raft_consensus.cc:504] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.744362 21438 leader_election.cc:288] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785), 02660991fa2c43d3ab2b7c758edacf0b (127.19.74.193:34723)
I20260806 22:42:13.747164 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.744616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:43597 (local address 127.19.74.195:32785)
0806 22:42:13.744669 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.744671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.744683 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:42:13.744720 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.744720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.744731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.744789 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:13.744970 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.745445 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.746401 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.746784 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.746873 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.746882 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.747056 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.747096 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:42:13.747164 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.744546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:32785 (local address 127.19.74.194:43597)
0806 22:42:13.744608 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.744621 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.744668 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.744822 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.744823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.744825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:13.744890 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.744892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.745534 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.745535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.746299 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.746302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.746869 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.746870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.746975 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.746978 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.747048 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.747090 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:42:13.747350 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.744759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:52177 (local address 127.19.74.193:34723)
0806 22:42:13.744822 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.744824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.744904 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:42:13.744978 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.744979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.744988 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.745058 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:13.745222 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.745679 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.746595 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.747002 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.747084 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.747100 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.747247 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.747286 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:42:13.747364 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.744634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:34723 (local address 127.19.74.194:52177)
0806 22:42:13.744832 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.744842 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.744915 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.745063 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.745063 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.745065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:13.745152 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.745154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.745762 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.745763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.746504 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.746505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.747096 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.747097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.747175 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.747178 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.747242 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.747280 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":129,"threads_started":1}
I20260806 22:42:13.747516 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdbfdba15c449dc939080105f40e909" candidate_uuid: "97d24554bfdd450098cd2490fab0fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0342d83e700845ce8506ca61be313acd" is_pre_election: true
I20260806 22:42:13.747701 21362 raft_consensus.cc:2454] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d24554bfdd450098cd2490fab0fc8e in term 0.
I20260806 22:42:13.747761 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdbfdba15c449dc939080105f40e909" candidate_uuid: "97d24554bfdd450098cd2490fab0fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02660991fa2c43d3ab2b7c758edacf0b" is_pre_election: true
I20260806 22:42:13.747934 21062 raft_consensus.cc:2454] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d24554bfdd450098cd2490fab0fc8e in term 0.
I20260806 22:42:13.747972 21150 leader_election.cc:302] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0342d83e700845ce8506ca61be313acd, 97d24554bfdd450098cd2490fab0fc8e; no voters:
I20260806 22:42:13.748121 21438 raft_consensus.cc:2781] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:13.748178 21438 raft_consensus.cc:482] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:13.748210 21438 raft_consensus.cc:3037] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.749148 21438 raft_consensus.cc:504] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.749285 21438 leader_election.cc:288] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785), 02660991fa2c43d3ab2b7c758edacf0b (127.19.74.193:34723)
I20260806 22:42:13.749439 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdbfdba15c449dc939080105f40e909" candidate_uuid: "97d24554bfdd450098cd2490fab0fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0342d83e700845ce8506ca61be313acd"
I20260806 22:42:13.749513 21362 raft_consensus.cc:3037] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.750298 21362 raft_consensus.cc:2454] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d24554bfdd450098cd2490fab0fc8e in term 1.
I20260806 22:42:13.750455 21150 leader_election.cc:302] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0342d83e700845ce8506ca61be313acd, 97d24554bfdd450098cd2490fab0fc8e; no voters:
I20260806 22:42:13.750430 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdbfdba15c449dc939080105f40e909" candidate_uuid: "97d24554bfdd450098cd2490fab0fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02660991fa2c43d3ab2b7c758edacf0b"
I20260806 22:42:13.750558 21438 raft_consensus.cc:2781] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:13.750535 21062 raft_consensus.cc:3037] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.750761 21438 raft_consensus.cc:686] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 97d24554bfdd450098cd2490fab0fc8e, State: Running, Role: LEADER
I20260806 22:42:13.750872 21438 consensus_queue.cc:237] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } }
I20260806 22:42:13.751471 21062 raft_consensus.cc:2454] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d24554bfdd450098cd2490fab0fc8e in term 1.
I20260806 22:42:13.751679 20912 catalog_manager.cc:5495] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97d24554bfdd450098cd2490fab0fc8e (127.19.74.194). New cstate: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:13.769305 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.766869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39770 (local address 127.19.74.194:45549)
0806 22:42:13.766934 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.766936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.766977 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:42:13.767058 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.767059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.767076 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.767143 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:13.767315 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.767840 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.768544 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.768738 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.768787 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.768972 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:13.769059 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:13.769121 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.769162 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.769179 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:42:13.771129 21062 raft_consensus.cc:1261] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Refusing update from remote peer 97d24554bfdd450098cd2490fab0fc8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:13.771160 21362 raft_consensus.cc:1261] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Refusing update from remote peer 97d24554bfdd450098cd2490fab0fc8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 22:42:13.771548 21124 tablet.cc:2369] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:13.771518 21441 consensus_queue.cc:1035] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:13.771710 21438 consensus_queue.cc:1035] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:13.774148 21444 mvcc.cc:205] Tried to move back new op lower bound from 7315685923923640320 to 7315685923844018176. Current Snapshot: MvccSnapshot[applied={T|T < 7315685923923640320}]
I20260806 22:42:13.775175 21446 mvcc.cc:205] Tried to move back new op lower bound from 7315685923923640320 to 7315685923844018176. Current Snapshot: MvccSnapshot[applied={T|T < 7315685923923640320}]
I20260806 22:42:13.778156 20912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:13.778251 20912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:13.779043 20912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6: Sending DeleteTablet for 3 replicas of tablet 1bdbfdba15c449dc939080105f40e909
I20260806 22:42:13.779423 21199 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdbfdba15c449dc939080105f40e909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:13 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:39762
I20260806 22:42:13.779434 21055 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdbfdba15c449dc939080105f40e909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:13 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:37202
I20260806 22:42:13.779556 21342 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdbfdba15c449dc939080105f40e909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:13 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59286
I20260806 22:42:13.779637 21455 tablet_replica.cc:330] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: stopping tablet replica
I20260806 22:42:13.779723 21455 raft_consensus.cc:2229] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:13.779822 21455 raft_consensus.cc:2258] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:13.779824 21456 tablet_replica.cc:330] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: stopping tablet replica
I20260806 22:42:13.779958 21456 raft_consensus.cc:2229] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:13.779965 20912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:13.780109 20912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:13.780295 21455 ts_tablet_manager.cc:1905] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:13.780328 21456 raft_consensus.cc:2258] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:13.780871 21456 ts_tablet_manager.cc:1905] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:13.781425 21342 tablet_service.cc:1467] Processing CreateTablet for tablet a166d4f553f04182a6a32dd572be27b6 (DEFAULT_TABLE table=test-table [id=da5594a36b4d4e4e9721b843da1a0b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:13.781551 21342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a166d4f553f04182a6a32dd572be27b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.782359 21455 ts_tablet_manager.cc:1918] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:13.782437 21455 log.cc:1196] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/1bdbfdba15c449dc939080105f40e909
I20260806 22:42:13.782727 21455 ts_tablet_manager.cc:1939] T 1bdbfdba15c449dc939080105f40e909 P 97d24554bfdd450098cd2490fab0fc8e: Deleting consensus metadata
I20260806 22:42:13.782876 21431 tablet_bootstrap.cc:492] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: Bootstrap starting.
I20260806 22:42:13.782904 21454 tablet_replica.cc:330] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: stopping tablet replica
I20260806 22:42:13.782991 21454 raft_consensus.cc:2229] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:13.783085 21454 raft_consensus.cc:2258] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:13.783253 21431 tablet_bootstrap.cc:654] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.783509 21454 ts_tablet_manager.cc:1905] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:13.783516 21456 ts_tablet_manager.cc:1918] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:13.783646 21456 log.cc:1196] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/1bdbfdba15c449dc939080105f40e909
I20260806 22:42:13.783931 21456 ts_tablet_manager.cc:1939] T 1bdbfdba15c449dc939080105f40e909 P 0342d83e700845ce8506ca61be313acd: Deleting consensus metadata
I20260806 22:42:13.783912 21055 tablet_service.cc:1467] Processing CreateTablet for tablet a166d4f553f04182a6a32dd572be27b6 (DEFAULT_TABLE table=test-table [id=da5594a36b4d4e4e9721b843da1a0b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:13.784017 21055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a166d4f553f04182a6a32dd572be27b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.785061 21454 ts_tablet_manager.cc:1918] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:13.785146 21454 log.cc:1196] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/1bdbfdba15c449dc939080105f40e909
I20260806 22:42:13.785290 21431 tablet_bootstrap.cc:492] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: No bootstrap required, opened a new log
I20260806 22:42:13.785354 21431 ts_tablet_manager.cc:1397] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:13.785382 21199 tablet_service.cc:1467] Processing CreateTablet for tablet a166d4f553f04182a6a32dd572be27b6 (DEFAULT_TABLE table=test-table [id=da5594a36b4d4e4e9721b843da1a0b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:13.785432 21454 ts_tablet_manager.cc:1939] T 1bdbfdba15c449dc939080105f40e909 P 02660991fa2c43d3ab2b7c758edacf0b: Deleting consensus metadata
I20260806 22:42:13.785494 21199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a166d4f553f04182a6a32dd572be27b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.785511 21431 raft_consensus.cc:348] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.785593 21431 raft_consensus.cc:374] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.785619 21431 raft_consensus.cc:729] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0342d83e700845ce8506ca61be313acd, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.785697 21431 consensus_queue.cc:260] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.785775 21431 ts_tablet_manager.cc:1428] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:13.786019 20900 catalog_manager.cc:4841] TS 97d24554bfdd450098cd2490fab0fc8e (127.19.74.194:45549): tablet 1bdbfdba15c449dc939080105f40e909 (table test-table [id=8d498db60cd94279a0b9ca1f6584f9e0]) successfully deleted
I20260806 22:42:13.786739 21433 tablet_bootstrap.cc:492] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: Bootstrap starting.
I20260806 22:42:13.786744 21432 tablet_bootstrap.cc:492] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: Bootstrap starting.
I20260806 22:42:13.787086 21432 tablet_bootstrap.cc:654] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.787142 21433 tablet_bootstrap.cc:654] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.787555 20900 catalog_manager.cc:4841] TS 02660991fa2c43d3ab2b7c758edacf0b (127.19.74.193:34723): tablet 1bdbfdba15c449dc939080105f40e909 (table test-table [id=8d498db60cd94279a0b9ca1f6584f9e0]) successfully deleted
I20260806 22:42:13.787863 20897 catalog_manager.cc:4841] TS 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785): tablet 1bdbfdba15c449dc939080105f40e909 (table test-table [id=8d498db60cd94279a0b9ca1f6584f9e0]) successfully deleted
I20260806 22:42:13.788087 21432 tablet_bootstrap.cc:492] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: No bootstrap required, opened a new log
I20260806 22:42:13.788162 21432 ts_tablet_manager.cc:1397] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:13.788422 21432 raft_consensus.cc:348] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.788480 21432 raft_consensus.cc:374] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.788494 21432 raft_consensus.cc:729] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02660991fa2c43d3ab2b7c758edacf0b, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.788563 21432 consensus_queue.cc:260] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.788686 21432 ts_tablet_manager.cc:1428] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:13.789099 21433 tablet_bootstrap.cc:492] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: No bootstrap required, opened a new log
I20260806 22:42:13.789146 21433 ts_tablet_manager.cc:1397] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:13.789280 21433 raft_consensus.cc:348] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.789379 21433 raft_consensus.cc:374] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.789409 21433 raft_consensus.cc:729] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d24554bfdd450098cd2490fab0fc8e, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.789460 21433 consensus_queue.cc:260] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.789572 21433 ts_tablet_manager.cc:1428] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:13.793812 21439 raft_consensus.cc:482] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:13.793881 21439 raft_consensus.cc:504] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.794198 21439 leader_election.cc:288] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97d24554bfdd450098cd2490fab0fc8e (127.19.74.194:45549), 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785)
I20260806 22:42:13.797858 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.794691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:53731 (local address 127.19.74.194:45549)
0806 22:42:13.794736 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.794738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.794763 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:42:13.794783 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.794784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.794792 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.794846 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:13.794988 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.795671 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.797149 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.797493 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.797578 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.797597 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.797756 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.797795 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:13.798120 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.794380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:45549 (local address 127.19.74.193:53731)
0806 22:42:13.794579 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.794587 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.794647 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.794854 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.794855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.794857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:13.794926 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.794928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.795748 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.795749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.796985 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.796988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.797597 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.797598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.797665 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.797670 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.797983 (+ 313us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.798027 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":45,"threads_started":1}
I20260806 22:42:13.798377 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a166d4f553f04182a6a32dd572be27b6" candidate_uuid: "02660991fa2c43d3ab2b7c758edacf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d24554bfdd450098cd2490fab0fc8e" is_pre_election: true
I20260806 22:42:13.798499 21220 raft_consensus.cc:2454] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02660991fa2c43d3ab2b7c758edacf0b in term 0.
I20260806 22:42:13.798735 21008 leader_election.cc:302] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02660991fa2c43d3ab2b7c758edacf0b, 97d24554bfdd450098cd2490fab0fc8e; no voters:
I20260806 22:42:13.798894 21439 raft_consensus.cc:2781] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:13.798933 21439 raft_consensus.cc:482] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:13.798960 21439 raft_consensus.cc:3037] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.799309 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.794346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:32785 (local address 127.19.74.193:32853)
0806 22:42:13.794449 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.794460 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.794512 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.794776 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.794777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.794779 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:13.794855 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.794857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.797096 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.797096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.798205 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.798208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.799054 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.799055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.799132 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.799135 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.799189 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.799225 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":64,"threads_started":1}
I20260806 22:42:13.799314 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.794491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:32853 (local address 127.19.74.195:32785)
0806 22:42:13.794540 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.794542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.794551 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:13.794574 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.794575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.794584 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.794748 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:13.794915 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.795667 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.798308 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.798936 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.798985 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.798992 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.799211 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.799253 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:42:13.799535 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a166d4f553f04182a6a32dd572be27b6" candidate_uuid: "02660991fa2c43d3ab2b7c758edacf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0342d83e700845ce8506ca61be313acd" is_pre_election: true
I20260806 22:42:13.799616 21362 raft_consensus.cc:2454] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02660991fa2c43d3ab2b7c758edacf0b in term 0.
I20260806 22:42:13.799793 21439 raft_consensus.cc:504] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.799929 21439 leader_election.cc:288] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [CANDIDATE]: Term 1 election: Requested vote from peers 97d24554bfdd450098cd2490fab0fc8e (127.19.74.194:45549), 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785)
I20260806 22:42:13.800068 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a166d4f553f04182a6a32dd572be27b6" candidate_uuid: "02660991fa2c43d3ab2b7c758edacf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d24554bfdd450098cd2490fab0fc8e"
I20260806 22:42:13.800108 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a166d4f553f04182a6a32dd572be27b6" candidate_uuid: "02660991fa2c43d3ab2b7c758edacf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0342d83e700845ce8506ca61be313acd"
I20260806 22:42:13.800148 21220 raft_consensus.cc:3037] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.800166 21362 raft_consensus.cc:3037] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.800860 21362 raft_consensus.cc:2454] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02660991fa2c43d3ab2b7c758edacf0b in term 1.
I20260806 22:42:13.801054 21006 leader_election.cc:302] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02660991fa2c43d3ab2b7c758edacf0b, 0342d83e700845ce8506ca61be313acd; no voters:
I20260806 22:42:13.801165 21439 raft_consensus.cc:2781] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:13.801209 21220 raft_consensus.cc:2454] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02660991fa2c43d3ab2b7c758edacf0b in term 1.
I20260806 22:42:13.801281 21439 raft_consensus.cc:686] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 LEADER]: Becoming Leader. State: Replica: 02660991fa2c43d3ab2b7c758edacf0b, State: Running, Role: LEADER
I20260806 22:42:13.801340 21439 consensus_queue.cc:237] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.802017 20912 catalog_manager.cc:5495] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 02660991fa2c43d3ab2b7c758edacf0b (127.19.74.193). New cstate: current_term: 1 leader_uuid: "02660991fa2c43d3ab2b7c758edacf0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } health_report { overall_health: UNKNOWN } } }
W20260806 22:42:13.828181 21409 tablet.cc:2369] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:13.846880 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.839489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.19.74.254:36997)
0806 22:42:13.839580 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.839582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.839739 (+ 157us) server_negotiation.cc:407] Connection header received
0806 22:42:13.839811 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.839811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.839822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.839894 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.840399 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.840893 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.841677 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.841856 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.841941 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.844258 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.844259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.844259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.844266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.845599 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.846032 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.846032 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.846032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.846080 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.846286 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.846286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.846286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.846379 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:13.846426 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.846674 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.846784 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":56}
I20260806 22:42:13.846882 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.839416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:36997 (local address 127.0.0.1:46464)
0806 22:42:13.839668 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.839676 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.839759 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.839898 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.839899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.840257 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:13.840307 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.840310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.840980 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.840981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.841571 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.841573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.841956 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.841957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.842013 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.842561 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:13.842566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:13.844122 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:13.845678 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.845685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.845687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.845951 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.846145 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:13.846146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:13.846147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:13.846222 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:13.846443 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:13.846450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:13.846506 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.846583 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.846737 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":122,"threads_started":1}
I20260806 22:42:13.849521 20912 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:46464:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:42:13.849771 20912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:13.851359 21055 tablet_service.cc:1467] Processing CreateTablet for tablet a97d4733b6ae48e0bab7015ca87233af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae7b046e759642d6b297e6f0f7b2027e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:13.851380 21342 tablet_service.cc:1467] Processing CreateTablet for tablet a97d4733b6ae48e0bab7015ca87233af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae7b046e759642d6b297e6f0f7b2027e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:13.851367 21199 tablet_service.cc:1467] Processing CreateTablet for tablet a97d4733b6ae48e0bab7015ca87233af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae7b046e759642d6b297e6f0f7b2027e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:13.851531 21342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97d4733b6ae48e0bab7015ca87233af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.851529 21199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97d4733b6ae48e0bab7015ca87233af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.851532 21055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97d4733b6ae48e0bab7015ca87233af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:13.854976 21431 tablet_bootstrap.cc:492] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: Bootstrap starting.
I20260806 22:42:13.855388 21431 tablet_bootstrap.cc:654] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.856058 21431 tablet_bootstrap.cc:492] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: No bootstrap required, opened a new log
I20260806 22:42:13.856105 21431 ts_tablet_manager.cc:1397] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:13.856232 21431 raft_consensus.cc:348] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.856287 21431 raft_consensus.cc:374] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.856302 21431 raft_consensus.cc:729] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0342d83e700845ce8506ca61be313acd, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.856331 21431 consensus_queue.cc:260] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.856393 21431 ts_tablet_manager.cc:1428] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:13.856500 21437 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.856552 21437 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:13.857299 21433 tablet_bootstrap.cc:492] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: Bootstrap starting.
I20260806 22:42:13.857345 21432 tablet_bootstrap.cc:492] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: Bootstrap starting.
I20260806 22:42:13.857826 21433 tablet_bootstrap.cc:654] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.857825 21432 tablet_bootstrap.cc:654] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:13.858728 21432 tablet_bootstrap.cc:492] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: No bootstrap required, opened a new log
I20260806 22:42:13.858788 21432 ts_tablet_manager.cc:1397] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:13.858861 21433 tablet_bootstrap.cc:492] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: No bootstrap required, opened a new log
I20260806 22:42:13.858911 21433 ts_tablet_manager.cc:1397] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:13.858932 21432 raft_consensus.cc:348] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.859021 21432 raft_consensus.cc:374] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.859047 21432 raft_consensus.cc:729] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02660991fa2c43d3ab2b7c758edacf0b, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.859102 21432 consensus_queue.cc:260] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.859125 21433 raft_consensus.cc:348] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.859187 21433 raft_consensus.cc:374] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:13.859192 21432 ts_tablet_manager.cc:1428] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:13.859212 21433 raft_consensus.cc:729] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d24554bfdd450098cd2490fab0fc8e, State: Initialized, Role: FOLLOWER
I20260806 22:42:13.859252 21433 consensus_queue.cc:260] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.859263 21439 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.859329 21433 ts_tablet_manager.cc:1428] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:13.859356 21439 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:13.859450 21449 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.859526 21449 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:13.932854 21449 raft_consensus.cc:482] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:13.932953 21449 raft_consensus.cc:504] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.933116 21449 leader_election.cc:288] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02660991fa2c43d3ab2b7c758edacf0b (127.19.74.193:34723), 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785)
I20260806 22:42:13.933367 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97d4733b6ae48e0bab7015ca87233af" candidate_uuid: "97d24554bfdd450098cd2490fab0fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02660991fa2c43d3ab2b7c758edacf0b" is_pre_election: true
I20260806 22:42:13.933385 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97d4733b6ae48e0bab7015ca87233af" candidate_uuid: "97d24554bfdd450098cd2490fab0fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0342d83e700845ce8506ca61be313acd" is_pre_election: true
I20260806 22:42:13.933511 21362 raft_consensus.cc:2454] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d24554bfdd450098cd2490fab0fc8e in term 0.
I20260806 22:42:13.933550 21062 raft_consensus.cc:2454] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d24554bfdd450098cd2490fab0fc8e in term 0.
I20260806 22:42:13.933692 21150 leader_election.cc:302] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0342d83e700845ce8506ca61be313acd, 97d24554bfdd450098cd2490fab0fc8e; no voters:
I20260806 22:42:13.933820 21449 raft_consensus.cc:2781] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:13.933864 21449 raft_consensus.cc:482] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:13.933894 21449 raft_consensus.cc:3037] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.934686 21449 raft_consensus.cc:504] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.934832 21449 leader_election.cc:288] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 02660991fa2c43d3ab2b7c758edacf0b (127.19.74.193:34723), 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785)
I20260806 22:42:13.935051 21062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97d4733b6ae48e0bab7015ca87233af" candidate_uuid: "97d24554bfdd450098cd2490fab0fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02660991fa2c43d3ab2b7c758edacf0b"
I20260806 22:42:13.935030 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97d4733b6ae48e0bab7015ca87233af" candidate_uuid: "97d24554bfdd450098cd2490fab0fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0342d83e700845ce8506ca61be313acd"
I20260806 22:42:13.935128 21362 raft_consensus.cc:3037] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.935148 21062 raft_consensus.cc:3037] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:13.935772 21062 raft_consensus.cc:2454] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d24554bfdd450098cd2490fab0fc8e in term 1.
I20260806 22:42:13.935825 21362 raft_consensus.cc:2454] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d24554bfdd450098cd2490fab0fc8e in term 1.
I20260806 22:42:13.936007 21152 leader_election.cc:302] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02660991fa2c43d3ab2b7c758edacf0b, 97d24554bfdd450098cd2490fab0fc8e; no voters:
I20260806 22:42:13.936090 21449 raft_consensus.cc:2781] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:13.936141 21449 raft_consensus.cc:686] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 97d24554bfdd450098cd2490fab0fc8e, State: Running, Role: LEADER
I20260806 22:42:13.936210 21449 consensus_queue.cc:237] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } }
I20260806 22:42:13.936493 21441 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: TxnStatusTablet state changed. Reason: New leader 97d24554bfdd450098cd2490fab0fc8e. Latest consensus state: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.936581 21441 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:13.936728 21475 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:42:13.936852 21475 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:42:13.936858 20912 catalog_manager.cc:5495] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97d24554bfdd450098cd2490fab0fc8e (127.19.74.194). New cstate: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:13.949081 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.946721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39784 (local address 127.19.74.194:45549)
0806 22:42:13.946785 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.946787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.946797 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:13.946868 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.946869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.946885 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.946960 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:13.947104 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.947589 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.948355 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.948548 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.948640 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.948823 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:13.948866 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:13.948928 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.948992 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.949013 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:42:13.949121 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.946635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:45549 (local address 127.0.0.1:39784)
0806 22:42:13.946707 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.946720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.946798 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.946972 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.946973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.946975 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:13.947025 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.947028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.947684 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.947686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.948266 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.948268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.948644 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.948645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.948721 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.948737 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:13.948930 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:13.948940 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.948989 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.949039 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:42:13.950241 21362 raft_consensus.cc:1261] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Refusing update from remote peer 97d24554bfdd450098cd2490fab0fc8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:13.950241 21062 raft_consensus.cc:1261] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Refusing update from remote peer 97d24554bfdd450098cd2490fab0fc8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:13.950449 21441 consensus_queue.cc:1035] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:13.950537 21449 consensus_queue.cc:1035] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:13.951232 21439 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: TxnStatusTablet state changed. Reason: New leader 97d24554bfdd450098cd2490fab0fc8e. Latest consensus state: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.951326 21439 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:13.951452 21437 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: TxnStatusTablet state changed. Reason: New leader 97d24554bfdd450098cd2490fab0fc8e. Latest consensus state: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.951542 21437 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:13.951704 21441 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.951784 21441 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:13.952250 21437 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.952348 21437 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 0342d83e700845ce8506ca61be313acd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:13.952421 21438 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.952502 21438 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 97d24554bfdd450098cd2490fab0fc8e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:13.953193 21439 tablet_replica.cc:439] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97d24554bfdd450098cd2490fab0fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02660991fa2c43d3ab2b7c758edacf0b" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34723 } } peers { permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 } } peers { permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 } } }
I20260806 22:42:13.953275 21439 tablet_replica.cc:442] T a97d4733b6ae48e0bab7015ca87233af P 02660991fa2c43d3ab2b7c758edacf0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:13.956537 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.954135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.19.74.193:34723)
0806 22:42:13.954200 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.954203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.954211 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:13.954249 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.954250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.954269 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.954354 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:13.954494 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.955026 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.955747 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.955954 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.956036 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.956206 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:13.956313 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:13.956394 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.956438 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.956456 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:42:13.964921 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.958965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:50199 (local address 127.19.74.194:45549)
0806 22:42:13.959024 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.959026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.962800 (+ 3774us) server_negotiation.cc:407] Connection header received
0806 22:42:13.962869 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.962870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.962881 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.962924 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:13.963064 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.963483 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.964218 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.964416 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.964497 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.964676 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:13.964708 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:13.964773 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.964834 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.964854 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:42:13.964964 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.958906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:45549 (local address 127.19.74.193:50199)
0806 22:42:13.962732 (+ 3826us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.962743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.962812 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.962924 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.962925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.962926 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:13.962994 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.962995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.963557 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.963558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.964102 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.964104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.964500 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.964502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.964568 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.964589 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:13.964775 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:13.964777 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.964829 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.964873 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3783,"thread_start_us":95,"threads_started":1}
I20260806 22:42:13.967180 21362 raft_consensus.cc:1261] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Refusing update from remote peer 02660991fa2c43d3ab2b7c758edacf0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:13.967303 21220 raft_consensus.cc:1261] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Refusing update from remote peer 02660991fa2c43d3ab2b7c758edacf0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:13.967418 21462 consensus_queue.cc:1035] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0342d83e700845ce8506ca61be313acd" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:13.967540 21439 consensus_queue.cc:1035] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "97d24554bfdd450098cd2490fab0fc8e" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:13.968982 21445 mvcc.cc:205] Tried to move back new op lower bound from 7315685924727840768 to 7315685924050796544. Current Snapshot: MvccSnapshot[applied={T|T < 7315685924727840768}]
I20260806 22:42:13.977558 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.973201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:34723 (local address 127.19.74.194:40949)
0806 22:42:13.974735 (+ 1534us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:13.974748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:13.974812 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:13.974944 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:13.974945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:13.974947 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:13.975008 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.975009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.975806 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.975807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.976739 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:13.976742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.977061 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.977062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:13.977135 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.977150 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:13.977372 (+ 222us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:13.977375 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:13.977417 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:42:13.977485 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1488,"thread_start_us":89,"threads_started":1}
I20260806 22:42:13.977577 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.973262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:40949 (local address 127.19.74.193:34723)
0806 22:42:13.973309 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.973311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.974814 (+ 1503us) server_negotiation.cc:407] Connection header received
0806 22:42:13.974865 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.974866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.974877 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.974940 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:13.975158 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.975739 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.976816 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.976995 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.977040 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.977274 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:13.977312 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:13.977368 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:13.977502 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 22:42:13.977521 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:42:13.986115 20913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:13.986243 20913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:13.986898 20913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44962028c32b438985d05c7c89d534c6: Sending DeleteTablet for 3 replicas of tablet a166d4f553f04182a6a32dd572be27b6
I20260806 22:42:13.987164 21342 tablet_service.cc:1514] Processing DeleteTablet for tablet a166d4f553f04182a6a32dd572be27b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:13 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59286
I20260806 22:42:13.987248 21456 tablet_replica.cc:330] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: stopping tablet replica
I20260806 22:42:13.987293 21456 raft_consensus.cc:2229] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:13.987323 21456 raft_consensus.cc:2258] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:13.987485 21199 tablet_service.cc:1514] Processing DeleteTablet for tablet a166d4f553f04182a6a32dd572be27b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:13 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:39762
I20260806 22:42:13.987596 21455 tablet_replica.cc:330] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: stopping tablet replica
I20260806 22:42:13.987633 21455 raft_consensus.cc:2229] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:13.987659 21455 raft_consensus.cc:2258] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:13.987860 21055 tablet_service.cc:1514] Processing DeleteTablet for tablet a166d4f553f04182a6a32dd572be27b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:13 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:37202
I20260806 22:42:13.987946 21454 tablet_replica.cc:330] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: stopping tablet replica
I20260806 22:42:13.987948 21456 ts_tablet_manager.cc:1905] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:13.987948 21455 ts_tablet_manager.cc:1905] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:13.987988 21454 raft_consensus.cc:2229] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:13.988059 21454 raft_consensus.cc:2258] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:13.989643 21455 ts_tablet_manager.cc:1918] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:13.989660 21456 ts_tablet_manager.cc:1918] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:13.989655 21454 ts_tablet_manager.cc:1905] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:13.989701 21455 log.cc:1196] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/a166d4f553f04182a6a32dd572be27b6
I20260806 22:42:13.989705 21456 log.cc:1196] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/a166d4f553f04182a6a32dd572be27b6
I20260806 22:42:13.989941 21456 ts_tablet_manager.cc:1939] T a166d4f553f04182a6a32dd572be27b6 P 0342d83e700845ce8506ca61be313acd: Deleting consensus metadata
I20260806 22:42:13.989940 21455 ts_tablet_manager.cc:1939] T a166d4f553f04182a6a32dd572be27b6 P 97d24554bfdd450098cd2490fab0fc8e: Deleting consensus metadata
I20260806 22:42:13.990279 20900 catalog_manager.cc:4841] TS 97d24554bfdd450098cd2490fab0fc8e (127.19.74.194:45549): tablet a166d4f553f04182a6a32dd572be27b6 (table test-table [id=da5594a36b4d4e4e9721b843da1a0b19]) successfully deleted
I20260806 22:42:13.990353 20897 catalog_manager.cc:4841] TS 0342d83e700845ce8506ca61be313acd (127.19.74.195:32785): tablet a166d4f553f04182a6a32dd572be27b6 (table test-table [id=da5594a36b4d4e4e9721b843da1a0b19]) successfully deleted
I20260806 22:42:13.991171 21454 ts_tablet_manager.cc:1918] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:13.991219 21454 log.cc:1196] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/a166d4f553f04182a6a32dd572be27b6
I20260806 22:42:13.991413 21454 ts_tablet_manager.cc:1939] T a166d4f553f04182a6a32dd572be27b6 P 02660991fa2c43d3ab2b7c758edacf0b: Deleting consensus metadata
I20260806 22:42:13.991854 20900 catalog_manager.cc:4841] TS 02660991fa2c43d3ab2b7c758edacf0b (127.19.74.193:34723): tablet a166d4f553f04182a6a32dd572be27b6 (table test-table [id=da5594a36b4d4e4e9721b843da1a0b19]) successfully deleted
Aug 06 22:42:13 dist-test-slave-pgkr krb5kdc[20867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:14.000607 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:13.990092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.19.74.193:34723)
0806 22:42:13.990404 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:13.990406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:13.990682 (+ 276us) server_negotiation.cc:407] Connection header received
0806 22:42:13.990768 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:13.990768 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:13.990777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:13.990801 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:13.991478 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.992220 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.993333 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:13.993651 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:13.993726 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:13.997192 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:13.997201 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:13.997201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:13.997208 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:13.999228 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.999647 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.999651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.999651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:13.999715 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:13.999966 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:13.999967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:13.999967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:14.000160 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:14.000300 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:14.000387 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:42:14.000469 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
W20260806 22:42:14.001035 21117 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37216
I20260806 22:42:14.009332 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:14.001940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46478 (local address 127.19.74.254:36997)
0806 22:42:14.001982 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:14.001984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:14.002176 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:42:14.002208 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:14.002209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:14.002220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:14.002286 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:14.002916 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:14.003630 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:14.004625 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:14.004930 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:14.005003 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:14.006325 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:14.006327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:14.006327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:14.006334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:14.008115 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:14.008492 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:14.008493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:14.008493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:14.008562 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:14.008790 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:14.008791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:14.008791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:14.008921 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:14.009025 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:14.009146 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:42:14.009221 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:42:14.009534 20913 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46478
I20260806 22:42:14.010221 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 20988
I20260806 22:42:14.020081 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 21132
I20260806 22:42:14.027865 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 21275
I20260806 22:42:14.033392 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 20880
2026-08-06T22:42:14Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2273 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:14 dist-test-slave-pgkr krb5kdc[21496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:14 dist-test-slave-pgkr krb5kdc[21496](info): set up 2 sockets
Aug 06 22:42:14 dist-test-slave-pgkr krb5kdc[21496](info): commencing operation
krb5kdc: starting...
W20260806 22:42:15.104445 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:15.221488 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:46127
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:46127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:15.306020 21510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:15.306218 21510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:15.306241 21510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:15.307442 21510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:15.307515 21510 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:42:15.307530 21510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:15.307542 21510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:15.307554 21510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:15.307566 21510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:15.308957 21510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:46127
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:46127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:15.309144 21510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:15.309301 21510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:15.314083 21516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:15.314097 21515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:15.314106 21519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:15.314389 21510 server_base.cc:1034] running on GCE node
I20260806 22:42:15.314702 21510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:15.314990 21510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:15.316143 21510 hybrid_clock.cc:648] HybridClock initialized: now 1786056135316123 us; error 39 us; skew 500 ppm
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:15.318745 21510 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:15.319155 21510 webserver.cc:460] Webserver started at http://127.19.74.254:38075/ using document root <none> and password file <none>
I20260806 22:42:15.319378 21510 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:15.319446 21510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:15.319603 21510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:15.320648 21510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "53d2d09585bc4ae08e5be00053c9013f"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "8061adbf648e7205d0f2703c5e7d7fb5"
server_key_iv: "01d71e8d7d1b5b1d6b57cff1397774b8"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.321023 21510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "53d2d09585bc4ae08e5be00053c9013f"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "8061adbf648e7205d0f2703c5e7d7fb5"
server_key_iv: "01d71e8d7d1b5b1d6b57cff1397774b8"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.322371 21510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:42:15.323123 21527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.323288 21510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.323390 21510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "53d2d09585bc4ae08e5be00053c9013f"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "8061adbf648e7205d0f2703c5e7d7fb5"
server_key_iv: "01d71e8d7d1b5b1d6b57cff1397774b8"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.323468 21510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:15.333094 21510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:15.335347 21510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:15.335512 21510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:15.340176 21510 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:46127
I20260806 22:42:15.340252 21593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:46127 every 8 connection(s)
I20260806 22:42:15.340561 21510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:15.341233 21594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:15.343410 21594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: Bootstrap starting.
I20260806 22:42:15.343941 21594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:15.344164 21594 log.cc:824] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:15.344823 21594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: No bootstrap required, opened a new log
I20260806 22:42:15.346051 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 21510
I20260806 22:42:15.346153 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:15.346318 19755 external_mini_cluster.cc:1428] Setting key aa4b87954ea4582ffad85a167457559f
I20260806 22:42:15.347126 21594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2d09585bc4ae08e5be00053c9013f" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46127 } }
I20260806 22:42:15.347280 21594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:15.347328 21594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d2d09585bc4ae08e5be00053c9013f, State: Initialized, Role: FOLLOWER
I20260806 22:42:15.347462 21594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2d09585bc4ae08e5be00053c9013f" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46127 } }
I20260806 22:42:15.347550 21594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:15.347584 21594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:15.347651 21594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:15.348294 21594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2d09585bc4ae08e5be00053c9013f" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46127 } }
I20260806 22:42:15.348390 21594 leader_election.cc:302] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53d2d09585bc4ae08e5be00053c9013f; no voters:
I20260806 22:42:15.348572 21594 leader_election.cc:288] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:15.348598 21599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:15.348702 21599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [term 1 LEADER]: Becoming Leader. State: Replica: 53d2d09585bc4ae08e5be00053c9013f, State: Running, Role: LEADER
I20260806 22:42:15.348780 21599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2d09585bc4ae08e5be00053c9013f" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46127 } }
I20260806 22:42:15.348834 21594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:15.349084 21599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53d2d09585bc4ae08e5be00053c9013f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2d09585bc4ae08e5be00053c9013f" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46127 } } }
I20260806 22:42:15.349160 21599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:15.349282 21599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53d2d09585bc4ae08e5be00053c9013f. Latest consensus state: current_term: 1 leader_uuid: "53d2d09585bc4ae08e5be00053c9013f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2d09585bc4ae08e5be00053c9013f" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46127 } } }
I20260806 22:42:15.349351 21599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:15.349491 21604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:15.350009 21604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:15.351564 21604 catalog_manager.cc:1347] Generated new cluster ID: eae68de8e1684c7cbd8703842e4922f7
I20260806 22:42:15.351622 21604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:15.357733 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.346837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44660 (local address 127.19.74.254:46127)
0806 22:42:15.347018 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.347030 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.347055 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:42:15.347144 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.347149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.347188 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.347337 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.348123 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.349372 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.350484 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.351155 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.351305 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.353992 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.353997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.353997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.354009 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.356043 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.356414 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.356418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.356419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.356545 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.356761 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.356761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.356762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.357163 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.357322 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.357399 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.357540 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260806 22:42:15.372574 21604 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:15.373296 21604 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:15.377480 21604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: Generated new TSK 0
I20260806 22:42:15.377660 21604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:15.403100 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:46127
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:15.484462 21620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:15.484664 21620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:15.484688 21620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:15.485746 21620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:15.485790 21620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:15.485809 21620 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:42:15.485821 21620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:15.485836 21620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:15.487186 21620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:46127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:15.487412 21620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:15.487604 21620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:15.492503 21626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:15.492635 21620 server_base.cc:1034] running on GCE node
W20260806 22:42:15.492517 21628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:15.492748 21625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:15.492970 21620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:15.493214 21620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:15.494417 21620 hybrid_clock.cc:648] HybridClock initialized: now 1786056135494413 us; error 37 us; skew 500 ppm
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:15.496873 21620 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:15.497305 21620 webserver.cc:460] Webserver started at http://127.19.74.193:36165/ using document root <none> and password file <none>
I20260806 22:42:15.497531 21620 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:15.497607 21620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:15.497746 21620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:15.498867 21620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "b748905c69b14519b5cfa9616fe3c028"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "e32edd4ce8cf1428223567c0144c593e"
server_key_iv: "8eeeecdc1b1f334ebd15f92c05250429"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.499231 21620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "b748905c69b14519b5cfa9616fe3c028"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "e32edd4ce8cf1428223567c0144c593e"
server_key_iv: "8eeeecdc1b1f334ebd15f92c05250429"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.500521 21620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:15.501236 21635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.501420 21620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:15.501519 21620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "b748905c69b14519b5cfa9616fe3c028"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "e32edd4ce8cf1428223567c0144c593e"
server_key_iv: "8eeeecdc1b1f334ebd15f92c05250429"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.501613 21620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:15.517278 21620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:15.519158 21620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:15.519311 21620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:15.519832 21620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:15.519898 21620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.519940 21620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:15.519970 21620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.526840 21620 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:37681
I20260806 22:42:15.526945 21757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:37681 every 8 connection(s)
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:15.527240 21620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:15.527906 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 21620
I20260806 22:42:15.528009 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:15.528205 19755 external_mini_cluster.cc:1428] Setting key c904f766c2e53e02081f4dea3e667314
I20260806 22:42:15.530447 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.520305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:52089 (local address 127.19.74.254:46127)
0806 22:42:15.520373 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.520376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.520579 (+ 203us) server_negotiation.cc:407] Connection header received
0806 22:42:15.521041 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.521043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.521062 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.521156 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.521957 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.522854 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.524395 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.524761 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.524845 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.527565 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.527567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.527567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.527575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.529051 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.529442 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.529444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.529444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.529508 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.529798 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.529799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.529799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.529916 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.529987 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.530269 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.530358 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:42:15.530649 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.520297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46127 (local address 127.19.74.193:52089)
0806 22:42:15.520447 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.520485 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.520942 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.521202 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.521205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.521547 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:15.521835 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.521842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.522955 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.522956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.524270 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.524273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.524854 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.524855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.524997 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.525546 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:15.525559 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:15.527485 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:15.529145 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.529153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.529155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.529367 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.529606 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.529607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.529607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.529718 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.529989 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:15.529997 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:15.530136 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.530245 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.530396 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260806 22:42:15.531133 21758 heartbeater.cc:347] Connected to a master server at 127.19.74.254:46127
I20260806 22:42:15.531240 21758 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:15.531427 21758 heartbeater.cc:511] Master 127.19.74.254:46127 requested a full tablet report, sending...
I20260806 22:42:15.531994 21543 ts_manager.cc:194] Registered new tserver with Master: b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681)
I20260806 22:42:15.532932 21543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:52089
I20260806 22:42:15.537846 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.532291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:47011 (local address 127.19.74.254:46127)
0806 22:42:15.532334 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.532336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.532419 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:42:15.532479 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.532480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.532487 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.532528 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.532841 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.533440 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.534130 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.534354 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.534431 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.535445 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.535446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.535446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.535452 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.536769 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.537180 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.537181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.537181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.537230 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.537429 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.537430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.537430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.537519 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.537567 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.537735 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.537791 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:42:15.537948 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.532188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46127 (local address 127.19.74.193:47011)
0806 22:42:15.532354 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.532366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.532426 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.532541 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.532541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.532726 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:15.532781 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.532782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.533532 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.533533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.534045 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.534046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.534430 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.534431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.534508 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.534948 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:15.534954 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:15.535354 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:15.536855 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.536856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.536857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.537100 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.537293 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.537294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.537295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.537367 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.537585 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:15.537586 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:15.537635 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.537721 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.537819 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:15.556769 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:46127
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:15.638499 21767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:15.638728 21767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:15.638753 21767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:15.639866 21767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:15.639909 21767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:15.639930 21767 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:42:15.639943 21767 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:15.639961 21767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:15.641309 21767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:46127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:15.641505 21767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:15.641682 21767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:15.646760 21775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:15.646795 21772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:15.646762 21773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:15.646807 21767 server_base.cc:1034] running on GCE node
I20260806 22:42:15.647187 21767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:15.647452 21767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:15.648600 21767 hybrid_clock.cc:648] HybridClock initialized: now 1786056135648598 us; error 34 us; skew 500 ppm
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:15.651234 21767 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:15.651669 21767 webserver.cc:460] Webserver started at http://127.19.74.194:46883/ using document root <none> and password file <none>
I20260806 22:42:15.651862 21767 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:15.651901 21767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:15.651990 21767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:15.652966 21767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "e31f95a2912643a5b27c656639c3c05b"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "9b8511603aecfee247ba954ee80da2ff"
server_key_iv: "48427f6370d9f40d763dedcd82ce49a6"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.653270 21767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "e31f95a2912643a5b27c656639c3c05b"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "9b8511603aecfee247ba954ee80da2ff"
server_key_iv: "48427f6370d9f40d763dedcd82ce49a6"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.654512 21767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:15.655282 21782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.655438 21767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:15.655546 21767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "e31f95a2912643a5b27c656639c3c05b"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "9b8511603aecfee247ba954ee80da2ff"
server_key_iv: "48427f6370d9f40d763dedcd82ce49a6"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.655622 21767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:15.664688 21767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:15.666504 21767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:15.666700 21767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:15.667239 21767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:15.667297 21767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.667335 21767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:15.667357 21767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:15.674593 21767 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:40513
I20260806 22:42:15.674672 21898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:40513 every 8 connection(s)
I20260806 22:42:15.674985 21767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:15.677058 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.667666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:38941 (local address 127.19.74.254:46127)
0806 22:42:15.667743 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.667745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.667925 (+ 180us) server_negotiation.cc:407] Connection header received
0806 22:42:15.668375 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.668377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.668391 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.668476 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.669097 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.669893 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.671261 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.671620 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.671708 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.674346 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.674347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.674348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.674357 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.675875 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.676302 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.676303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.676303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.676358 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.676562 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.676562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.676562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.676670 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.676721 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.676935 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.676996 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:42:15.677230 21792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.667639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46127 (local address 127.19.74.194:38941)
0806 22:42:15.667794 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.667832 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.668279 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.668540 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.668543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.668768 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:15.669001 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.669006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.670017 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.670018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.671162 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.671165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.671728 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.671729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.671851 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.672358 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:15.672368 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:15.674227 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:15.675947 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.675952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.675953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.676222 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.676415 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.676415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.676416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.676509 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.676724 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:15.676728 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:15.676828 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.676906 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.677018 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260806 22:42:15.677687 21899 heartbeater.cc:347] Connected to a master server at 127.19.74.254:46127
I20260806 22:42:15.677785 21899 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:15.677975 21899 heartbeater.cc:511] Master 127.19.74.254:46127 requested a full tablet report, sending...
I20260806 22:42:15.678331 21544 ts_manager.cc:194] Registered new tserver with Master: e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513)
I20260806 22:42:15.678861 21544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:38941
I20260806 22:42:15.681375 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 21767
I20260806 22:42:15.681473 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:15.681716 19755 external_mini_cluster.cc:1428] Setting key b1af3b4a10c6d4c86d90bf64c22788d5
I20260806 22:42:15.684322 21907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.678711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46127 (local address 127.19.74.194:51033)
0806 22:42:15.678854 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.678865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.678927 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.679081 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.679082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.679274 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:15.679326 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.679328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.680001 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.680001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.680489 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.680490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.680783 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.680783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.680837 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.681264 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:15.681269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:15.681595 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:15.683234 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.683235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.683235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.683468 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.683653 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.683654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.683654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.683726 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.683974 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:15.683974 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:15.684022 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.684100 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.684224 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260806 22:42:15.684358 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.678793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:51033 (local address 127.19.74.254:46127)
0806 22:42:15.678838 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.678841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.678962 (+ 121us) server_negotiation.cc:407] Connection header received
0806 22:42:15.679001 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.679002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.679013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.679057 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.679405 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.679918 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.680552 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.680723 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.680769 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.681700 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.681701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.681702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.681708 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.683155 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.683541 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.683542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.683542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.683590 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.683786 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.683787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.683787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.683911 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.683960 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.684150 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.684240 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:15.707355 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:46127
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:15.787735 21910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:15.787973 21910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:15.788012 21910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:15.789166 21910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:15.789239 21910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:15.789271 21910 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:42:15.789302 21910 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:15.789333 21910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:15.790766 21910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:46127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:15.791028 21910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:15.791255 21910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:15.796060 21916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:15.796070 21915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:15.796249 21910 server_base.cc:1034] running on GCE node
W20260806 22:42:15.796365 21918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:15.796553 21910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:15.796808 21910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:15.797974 21910 hybrid_clock.cc:648] HybridClock initialized: now 1786056135797956 us; error 28 us; skew 500 ppm
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:15.800436 21910 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:15.800889 21910 webserver.cc:460] Webserver started at http://127.19.74.195:45975/ using document root <none> and password file <none>
I20260806 22:42:15.801059 21910 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:15.801097 21910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:15.801186 21910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:15.802088 21910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "bd0db979e1df25f18bcd759e958db749"
server_key_iv: "980611182a7f18e3ab5c67403cee383b"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.802388 21910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "bd0db979e1df25f18bcd759e958db749"
server_key_iv: "980611182a7f18e3ab5c67403cee383b"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.803757 21910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:15.804527 21925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.804709 21910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:15.804796 21910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c"
format_stamp: "Formatted at 2026-08-06 22:42:15 on dist-test-slave-pgkr"
server_key: "bd0db979e1df25f18bcd759e958db749"
server_key_iv: "980611182a7f18e3ab5c67403cee383b"
server_key_version: "encryptionkey@0"
I20260806 22:42:15.804848 21910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:15.829914 21910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:15.832427 21910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:15.832544 21910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:15.833027 21910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:15.833066 21910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.833099 21910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:15.833149 21910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.839893 21910 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:39635
I20260806 22:42:15.839967 22043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:39635 every 8 connection(s)
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:15.840287 21910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:15.841907 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 21910
I20260806 22:42:15.842043 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:15.842262 19755 external_mini_cluster.cc:1428] Setting key 97279353cbf50fdba1e75fb4bfa79d63
I20260806 22:42:15.843637 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.833567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:54697 (local address 127.19.74.254:46127)
0806 22:42:15.833644 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.833647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.833833 (+ 186us) server_negotiation.cc:407] Connection header received
0806 22:42:15.834348 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.834350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.834364 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.834452 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.835368 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.836217 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.837725 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.838054 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.838136 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.840781 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.840783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.840783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.840791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.841989 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.842513 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.842514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.842515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.842581 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.842948 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.842949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.842950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.843079 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.843165 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.843466 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.843550 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:42:15.843830 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.833527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46127 (local address 127.19.74.195:54697)
0806 22:42:15.833704 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.833746 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.834243 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.834555 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.834561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.834977 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:15.835256 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.835263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.836319 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.836320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.837605 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.837618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.838136 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.838137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.838270 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.838794 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:15.838805 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:15.840700 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:15.842115 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.842122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.842125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.842432 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.842710 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.842711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.842711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.842865 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.843157 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:15.843164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:15.843318 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.843441 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.843601 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260806 22:42:15.844274 22044 heartbeater.cc:347] Connected to a master server at 127.19.74.254:46127
I20260806 22:42:15.844374 22044 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:15.844568 22044 heartbeater.cc:511] Master 127.19.74.254:46127 requested a full tablet report, sending...
I20260806 22:42:15.844977 21544 ts_manager.cc:194] Registered new tserver with Master: ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:15.845479 21544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:54697
I20260806 22:42:15.850601 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.845329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:37101 (local address 127.19.74.254:46127)
0806 22:42:15.845367 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.845369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.845495 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:42:15.845593 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.845594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.845605 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.845670 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.845967 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.846427 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.847262 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.847458 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.847548 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.848304 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.848305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.848305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.848312 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.849439 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.849792 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.849793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.849794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.849849 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.850111 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.850113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.850113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.850219 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.850295 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.850452 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.850517 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:42:15.850639 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.845274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46127 (local address 127.19.74.195:37101)
0806 22:42:15.845429 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.845438 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.845509 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.845676 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.845677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.845845 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:15.845893 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.845894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.846519 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.846520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.847167 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.847170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.847551 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.847552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.847623 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.847945 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:15.847948 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:15.848201 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:15.849536 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.849537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.849538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.849707 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.849916 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.849917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.849917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.849997 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.850287 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:15.850289 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:15.850332 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.850412 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.850515 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260806 22:42:15.855036 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:15.877966 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.871476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44662 (local address 127.19.74.254:46127)
0806 22:42:15.871539 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.871541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.871550 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:15.871582 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.871583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.871591 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.871642 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.872168 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.872666 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.873349 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.873532 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.873578 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.875679 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.875680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.875681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.875687 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.876817 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.877098 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.877099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.877099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.877157 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.877338 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.877339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.877340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.877508 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.877644 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.877778 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.877860 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:42:15.879679 21544 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:15.881170 21544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:15.887562 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.883587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.19.74.194:40513)
0806 22:42:15.883871 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.883875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.883885 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:15.883920 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.883922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.883937 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.884015 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:15.884263 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.885403 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.886577 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.887085 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.887202 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.887266 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.887417 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.887458 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":78,"threads_started":1}
I20260806 22:42:15.887578 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.883431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:39635 (local address 127.0.0.1:46834)
0806 22:42:15.883747 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.883759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.883815 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.884121 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.884122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.884124 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:15.884194 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.884196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.885154 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.885155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.886366 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.886369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.887281 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.887282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.887345 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.887350 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.887383 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.887459 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260806 22:42:15.887657 22062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.883624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:37681 (local address 127.0.0.1:58392)
0806 22:42:15.883749 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.883757 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.883796 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.884245 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.884246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.884248 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:15.884290 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.884291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.885569 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.885570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.886396 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.886399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.887410 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.887411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.887461 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.887464 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.887492 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.887527 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260806 22:42:15.887842 22064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.883541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.19.74.195:39635)
0806 22:42:15.883926 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.883932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.883941 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:15.883981 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.883983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.883997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.884101 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:15.884287 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.885046 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.886435 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.887173 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.887572 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.887652 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.887680 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.887728 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":92,"threads_started":1}
I20260806 22:42:15.887564 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.883404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:40513 (local address 127.0.0.1:54844)
0806 22:42:15.883575 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.883616 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.883679 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.884109 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.884112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.884114 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:15.884179 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.884181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.885490 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.885491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.886500 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.886501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.887207 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.887208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.887285 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.887297 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.887396 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.887463 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":22,"thread_start_us":49,"threads_started":1}
I20260806 22:42:15.888077 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.883846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.19.74.193:37681)
0806 22:42:15.884015 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.884019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.884028 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:15.884065 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.884067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.884081 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.884374 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:15.884452 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.885502 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.886486 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.887325 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.887827 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.887912 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.887942 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.887983 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260806 22:42:15.888522 21973 tablet_service.cc:1467] Processing CreateTablet for tablet 756e0e161c0043d1a56dffefa0c93613 (DEFAULT_TABLE table=test-table [id=4a588220a74645aa9c13acc573c5fb86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:15.888917 21973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 756e0e161c0043d1a56dffefa0c93613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:15.888984 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 756e0e161c0043d1a56dffefa0c93613 (DEFAULT_TABLE table=test-table [id=4a588220a74645aa9c13acc573c5fb86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:15.889338 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 756e0e161c0043d1a56dffefa0c93613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:15.891119 21833 tablet_service.cc:1467] Processing CreateTablet for tablet 756e0e161c0043d1a56dffefa0c93613 (DEFAULT_TABLE table=test-table [id=4a588220a74645aa9c13acc573c5fb86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:15.891467 21833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 756e0e161c0043d1a56dffefa0c93613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:15.892266 22068 tablet_bootstrap.cc:492] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: Bootstrap starting.
I20260806 22:42:15.892846 22069 tablet_bootstrap.cc:492] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: Bootstrap starting.
I20260806 22:42:15.893392 22069 tablet_bootstrap.cc:654] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:15.893500 22068 tablet_bootstrap.cc:654] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:15.893652 22069 log.cc:824] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:15.893918 22068 log.cc:824] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:15.894378 22069 tablet_bootstrap.cc:492] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: No bootstrap required, opened a new log
I20260806 22:42:15.894467 22069 ts_tablet_manager.cc:1397] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:15.894795 22068 tablet_bootstrap.cc:492] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: No bootstrap required, opened a new log
I20260806 22:42:15.894863 22068 ts_tablet_manager.cc:1397] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:42:15.895331 22072 tablet_bootstrap.cc:492] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Bootstrap starting.
I20260806 22:42:15.895941 22072 tablet_bootstrap.cc:654] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:15.896279 22072 log.cc:824] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:15.896222 22069 raft_consensus.cc:348] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.896384 22069 raft_consensus.cc:374] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:15.896425 22069 raft_consensus.cc:729] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b748905c69b14519b5cfa9616fe3c028, State: Initialized, Role: FOLLOWER
I20260806 22:42:15.896544 22069 consensus_queue.cc:260] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.896811 22069 ts_tablet_manager.cc:1428] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:42:15.896950 22072 tablet_bootstrap.cc:492] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: No bootstrap required, opened a new log
I20260806 22:42:15.897022 22072 ts_tablet_manager.cc:1397] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:15.897011 22068 raft_consensus.cc:348] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.897168 22068 raft_consensus.cc:374] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:15.897207 22068 raft_consensus.cc:729] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd6729f42fa43298b88a1ae3ad9fd5c, State: Initialized, Role: FOLLOWER
I20260806 22:42:15.897315 22068 consensus_queue.cc:260] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.897564 22068 ts_tablet_manager.cc:1428] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:42:15.897599 22044 heartbeater.cc:503] Master 127.19.74.254:46127 was elected leader, sending a full tablet report...
I20260806 22:42:15.898533 22072 raft_consensus.cc:348] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.898712 21758 heartbeater.cc:503] Master 127.19.74.254:46127 was elected leader, sending a full tablet report...
I20260806 22:42:15.898727 22072 raft_consensus.cc:374] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:15.898765 22072 raft_consensus.cc:729] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31f95a2912643a5b27c656639c3c05b, State: Initialized, Role: FOLLOWER
I20260806 22:42:15.898880 22072 consensus_queue.cc:260] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.899185 22072 ts_tablet_manager.cc:1428] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:15.899361 21899 heartbeater.cc:503] Master 127.19.74.254:46127 was elected leader, sending a full tablet report...
I20260806 22:42:15.908332 22076 raft_consensus.cc:482] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:15.908429 22076 raft_consensus.cc:504] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.908725 22076 leader_election.cc:288] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635), b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681)
I20260806 22:42:15.912372 22064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.908942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:50927 (local address 127.19.74.195:39635)
0806 22:42:15.908982 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.908984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.908994 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:15.909084 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.909085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.909093 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.909152 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:15.909300 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.910003 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.911422 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.911997 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.912087 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.912095 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.912261 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.912311 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:15.912372 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.908953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:55499 (local address 127.19.74.193:37681)
0806 22:42:15.908991 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.908993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.909077 (+ 84us) server_negotiation.cc:407] Connection header received
0806 22:42:15.909148 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.909149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.909157 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.909202 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:15.909341 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.909998 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.911421 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.911997 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.912087 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.912095 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.912261 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.912312 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:15.912400 21792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.908853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:39635 (local address 127.19.74.194:50927)
0806 22:42:15.908947 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.908958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.909002 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.909165 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.909166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.909168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:15.909232 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.909234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.910078 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.910079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.911331 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.911333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.912097 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.912097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.912182 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.912185 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.912246 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.912308 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":93}
I20260806 22:42:15.912420 22077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.908881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:37681 (local address 127.19.74.194:55499)
0806 22:42:15.909016 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.909024 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.909072 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.909214 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.909215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.909216 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:15.909278 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.909279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.910078 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.910079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.911331 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.911333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.912102 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.912103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.912170 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.912174 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.912244 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.912308 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":65,"thread_start_us":81,"threads_started":1}
I20260806 22:42:15.912758 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "756e0e161c0043d1a56dffefa0c93613" candidate_uuid: "e31f95a2912643a5b27c656639c3c05b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" is_pre_election: true
I20260806 22:42:15.912747 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "756e0e161c0043d1a56dffefa0c93613" candidate_uuid: "e31f95a2912643a5b27c656639c3c05b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b748905c69b14519b5cfa9616fe3c028" is_pre_election: true
I20260806 22:42:15.912876 21996 raft_consensus.cc:2454] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e31f95a2912643a5b27c656639c3c05b in term 0.
I20260806 22:42:15.912876 21710 raft_consensus.cc:2454] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e31f95a2912643a5b27c656639c3c05b in term 0.
I20260806 22:42:15.913113 21783 leader_election.cc:302] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e31f95a2912643a5b27c656639c3c05b, ecd6729f42fa43298b88a1ae3ad9fd5c; no voters:
I20260806 22:42:15.913255 22076 raft_consensus.cc:2781] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:15.913290 22076 raft_consensus.cc:482] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:15.913305 22076 raft_consensus.cc:3037] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:15.913857 22076 raft_consensus.cc:504] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.913987 22076 leader_election.cc:288] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [CANDIDATE]: Term 1 election: Requested vote from peers ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635), b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681)
I20260806 22:42:15.914161 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "756e0e161c0043d1a56dffefa0c93613" candidate_uuid: "e31f95a2912643a5b27c656639c3c05b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c"
I20260806 22:42:15.914161 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "756e0e161c0043d1a56dffefa0c93613" candidate_uuid: "e31f95a2912643a5b27c656639c3c05b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b748905c69b14519b5cfa9616fe3c028"
I20260806 22:42:15.914233 21710 raft_consensus.cc:3037] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:15.914233 21996 raft_consensus.cc:3037] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:15.915068 21996 raft_consensus.cc:2454] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e31f95a2912643a5b27c656639c3c05b in term 1.
I20260806 22:42:15.915068 21710 raft_consensus.cc:2454] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e31f95a2912643a5b27c656639c3c05b in term 1.
I20260806 22:42:15.915625 21784 leader_election.cc:302] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b748905c69b14519b5cfa9616fe3c028, e31f95a2912643a5b27c656639c3c05b; no voters:
I20260806 22:42:15.915745 22076 raft_consensus.cc:2781] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:15.915911 22076 raft_consensus.cc:686] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 1 LEADER]: Becoming Leader. State: Replica: e31f95a2912643a5b27c656639c3c05b, State: Running, Role: LEADER
I20260806 22:42:15.916010 22076 consensus_queue.cc:237] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } }
I20260806 22:42:15.916756 21543 catalog_manager.cc:5495] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b reported cstate change: term changed from 0 to 1, leader changed from <none> to e31f95a2912643a5b27c656639c3c05b (127.19.74.194). New cstate: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } health_report { overall_health: UNKNOWN } } }
W20260806 22:42:15.925683 21900 tablet.cc:2369] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:42:15 dist-test-slave-pgkr krb5kdc[21496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056135, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:15.945715 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.939328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46127 (local address 127.0.0.1:44666)
0806 22:42:15.939503 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.939520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.939594 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.939754 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.939755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.940039 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:15.940080 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.940082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.940674 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.940675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.941129 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.941130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.941449 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.941450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.941501 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.942041 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:15.942045 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:15.943299 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:15.944748 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.944754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.944756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.944941 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.945164 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:15.945165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:15.945165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:15.945223 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:15.945444 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:15.945450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:15.945491 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.945540 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.945644 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260806 22:42:15.945796 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.939404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.19.74.254:46127)
0806 22:42:15.939481 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.939483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.939581 (+ 98us) server_negotiation.cc:407] Connection header received
0806 22:42:15.939670 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.939671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.939681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.939749 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:15.940161 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.940589 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.941207 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.941371 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.941448 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.943373 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:15.943374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:15.943374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:15.943380 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:15.944674 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.945013 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.945018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.945018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.945087 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:15.945280 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:15.945280 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:15.945281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:15.945389 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:15.945439 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.945594 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.945691 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":47}
I20260806 22:42:15.947598 21543 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:44666:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:42:15.947799 21543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:15.949411 21973 tablet_service.cc:1467] Processing CreateTablet for tablet f98b26abc3984529b44d26dfe12b6dcc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c38bf56cfad46faa35a9ffe5b38c02a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:15.949414 21833 tablet_service.cc:1467] Processing CreateTablet for tablet f98b26abc3984529b44d26dfe12b6dcc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c38bf56cfad46faa35a9ffe5b38c02a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:15.949409 21688 tablet_service.cc:1467] Processing CreateTablet for tablet f98b26abc3984529b44d26dfe12b6dcc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c38bf56cfad46faa35a9ffe5b38c02a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:15.949579 21833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98b26abc3984529b44d26dfe12b6dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:15.949579 21973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98b26abc3984529b44d26dfe12b6dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:15.949579 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98b26abc3984529b44d26dfe12b6dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:15.950824 22068 tablet_bootstrap.cc:492] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: Bootstrap starting.
I20260806 22:42:15.951128 22072 tablet_bootstrap.cc:492] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: Bootstrap starting.
I20260806 22:42:15.951351 22069 tablet_bootstrap.cc:492] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: Bootstrap starting.
I20260806 22:42:15.951444 22068 tablet_bootstrap.cc:654] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:15.951565 22072 tablet_bootstrap.cc:654] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:15.952018 22069 tablet_bootstrap.cc:654] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:15.952252 22068 tablet_bootstrap.cc:492] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: No bootstrap required, opened a new log
I20260806 22:42:15.952312 22068 ts_tablet_manager.cc:1397] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:15.952481 22068 raft_consensus.cc:348] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.952557 22068 raft_consensus.cc:374] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:15.952584 22068 raft_consensus.cc:729] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd6729f42fa43298b88a1ae3ad9fd5c, State: Initialized, Role: FOLLOWER
I20260806 22:42:15.952630 22068 consensus_queue.cc:260] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.952726 22072 tablet_bootstrap.cc:492] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: No bootstrap required, opened a new log
I20260806 22:42:15.952742 22069 tablet_bootstrap.cc:492] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: No bootstrap required, opened a new log
I20260806 22:42:15.952741 22068 ts_tablet_manager.cc:1428] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:15.952781 22072 ts_tablet_manager.cc:1397] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:15.952782 22069 ts_tablet_manager.cc:1397] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:15.952854 22075 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.952929 22069 raft_consensus.cc:348] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.952960 22075 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:15.952926 22072 raft_consensus.cc:348] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.952992 22069 raft_consensus.cc:374] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:15.953007 22069 raft_consensus.cc:729] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b748905c69b14519b5cfa9616fe3c028, State: Initialized, Role: FOLLOWER
I20260806 22:42:15.953006 22072 raft_consensus.cc:374] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:15.953050 22072 raft_consensus.cc:729] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31f95a2912643a5b27c656639c3c05b, State: Initialized, Role: FOLLOWER
I20260806 22:42:15.953066 22069 consensus_queue.cc:260] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.953092 22072 consensus_queue.cc:260] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.953146 22069 ts_tablet_manager.cc:1428] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:15.953195 22072 ts_tablet_manager.cc:1428] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:15.953385 22076 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.953459 22076 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:15.953583 22074 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.953673 22074 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:15.975313 22076 raft_consensus.cc:482] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:15.975381 22076 raft_consensus.cc:504] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.975543 22076 leader_election.cc:288] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681), ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:15.975751 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98b26abc3984529b44d26dfe12b6dcc" candidate_uuid: "e31f95a2912643a5b27c656639c3c05b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b748905c69b14519b5cfa9616fe3c028" is_pre_election: true
I20260806 22:42:15.975751 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98b26abc3984529b44d26dfe12b6dcc" candidate_uuid: "e31f95a2912643a5b27c656639c3c05b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" is_pre_election: true
I20260806 22:42:15.975836 21710 raft_consensus.cc:2454] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e31f95a2912643a5b27c656639c3c05b in term 0.
I20260806 22:42:15.975836 21996 raft_consensus.cc:2454] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e31f95a2912643a5b27c656639c3c05b in term 0.
I20260806 22:42:15.975968 21783 leader_election.cc:302] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e31f95a2912643a5b27c656639c3c05b, ecd6729f42fa43298b88a1ae3ad9fd5c; no voters:
I20260806 22:42:15.976078 22076 raft_consensus.cc:2781] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:15.976127 22076 raft_consensus.cc:482] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:15.976168 22076 raft_consensus.cc:3037] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:15.976759 22076 raft_consensus.cc:504] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.976897 22076 leader_election.cc:288] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [CANDIDATE]: Term 1 election: Requested vote from peers b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681), ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:15.977065 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98b26abc3984529b44d26dfe12b6dcc" candidate_uuid: "e31f95a2912643a5b27c656639c3c05b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c"
I20260806 22:42:15.977099 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98b26abc3984529b44d26dfe12b6dcc" candidate_uuid: "e31f95a2912643a5b27c656639c3c05b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b748905c69b14519b5cfa9616fe3c028"
I20260806 22:42:15.977156 21996 raft_consensus.cc:3037] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:15.977177 21710 raft_consensus.cc:3037] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:15.977907 21710 raft_consensus.cc:2454] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e31f95a2912643a5b27c656639c3c05b in term 1.
I20260806 22:42:15.977912 21996 raft_consensus.cc:2454] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e31f95a2912643a5b27c656639c3c05b in term 1.
I20260806 22:42:15.978070 21784 leader_election.cc:302] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b748905c69b14519b5cfa9616fe3c028, e31f95a2912643a5b27c656639c3c05b; no voters:
I20260806 22:42:15.978195 22076 raft_consensus.cc:2781] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:15.978260 22076 raft_consensus.cc:686] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [term 1 LEADER]: Becoming Leader. State: Replica: e31f95a2912643a5b27c656639c3c05b, State: Running, Role: LEADER
I20260806 22:42:15.978296 22076 consensus_queue.cc:237] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:15.978662 22078 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: TxnStatusTablet state changed. Reason: New leader e31f95a2912643a5b27c656639c3c05b. Latest consensus state: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.978751 22078 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:15.978965 21543 catalog_manager.cc:5495] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b reported cstate change: term changed from 0 to 1, leader changed from <none> to e31f95a2912643a5b27c656639c3c05b (127.19.74.194). New cstate: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:15.979031 22091 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:42:15.979140 22091 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:42:15.991142 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.988903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.19.74.194:40513)
0806 22:42:15.988943 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.988945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.989026 (+ 81us) server_negotiation.cc:407] Connection header received
0806 22:42:15.989119 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.989120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.989142 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.989218 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:15.989317 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.989716 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.990362 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.990530 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.990616 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.990829 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:15.990921 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:15.991005 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:15.991052 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:42:15.991075 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:15.991165 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.988851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:40513 (local address 127.0.0.1:54848)
0806 22:42:15.988968 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:15.988979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:15.989039 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:15.989224 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:15.989225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:15.989226 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:15.989271 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.989273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.989795 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.989796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.990286 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:15.990288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.990652 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.990654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:15.990709 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.990730 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:15.990985 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:15.990995 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:42:15.991047 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:42:15.991099 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 22:42:15.992825 21996 raft_consensus.cc:1261] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Refusing update from remote peer e31f95a2912643a5b27c656639c3c05b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:15.992832 21710 raft_consensus.cc:1261] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Refusing update from remote peer e31f95a2912643a5b27c656639c3c05b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:15.993126 22076 consensus_queue.cc:1035] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:15.993296 22076 consensus_queue.cc:1035] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:15.994354 22075 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: TxnStatusTablet state changed. Reason: New leader e31f95a2912643a5b27c656639c3c05b. Latest consensus state: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.994496 22075 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:15.994979 22074 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: TxnStatusTablet state changed. Reason: New leader e31f95a2912643a5b27c656639c3c05b. Latest consensus state: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.995086 22074 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:15.995338 22078 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.995427 22078 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:15.995769 22076 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.995754 22075 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.995826 22076 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P e31f95a2912643a5b27c656639c3c05b: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:15.995826 22075 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P ecd6729f42fa43298b88a1ae3ad9fd5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:15.995998 22095 mvcc.cc:205] Tried to move back new op lower bound from 7315685933023981568 to 7315685932967649280. Current Snapshot: MvccSnapshot[applied={T|T < 7315685933023981568}]
I20260806 22:42:15.996270 22074 tablet_replica.cc:439] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e31f95a2912643a5b27c656639c3c05b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } } }
I20260806 22:42:15.996345 22074 tablet_replica.cc:442] T f98b26abc3984529b44d26dfe12b6dcc P b748905c69b14519b5cfa9616fe3c028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:15.996752 22094 mvcc.cc:205] Tried to move back new op lower bound from 7315685933023981568 to 7315685932967649280. Current Snapshot: MvccSnapshot[applied={T|T < 7315685933023981568 or (T in {7315685933023981568})}]
I20260806 22:42:15.997249 22096 mvcc.cc:205] Tried to move back new op lower bound from 7315685933023981568 to 7315685932967649280. Current Snapshot: MvccSnapshot[applied={T|T < 7315685933023981568}]
I20260806 22:42:16.000068 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:15.997762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54862 (local address 127.19.74.194:40513)
0806 22:42:15.997793 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:15.997795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:15.997802 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:15.997854 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:15.997855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:15.997862 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:15.997905 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:15.998012 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.998414 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.999513 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:15.999687 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:15.999737 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:15.999889 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:15.999912 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:15.999951 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:16.000013 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:42:16.000027 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 22:42:16.004598 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.002335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:48893 (local address 127.19.74.194:40513)
0806 22:42:16.002384 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:16.002385 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:16.002417 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:42:16.002479 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:16.002480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:16.002487 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:16.002537 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:16.002696 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.003108 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.003857 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.004063 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.004136 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.004343 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:16.004371 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:16.004449 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:16.004507 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 22:42:16.004523 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:42:16.004653 21907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.002282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:40513 (local address 127.19.74.194:48893)
0806 22:42:16.002378 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:16.002388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:16.002430 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:16.002540 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:16.002541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:16.002541 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:16.002645 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.002646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.003202 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.003203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.003765 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.003767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.004162 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.004163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.004220 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.004233 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:16.004449 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:16.004451 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:16.004504 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:42:16.004543 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:42:16.007100 21710 raft_consensus.cc:1261] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Refusing update from remote peer e31f95a2912643a5b27c656639c3c05b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:16.007135 21996 raft_consensus.cc:1261] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Refusing update from remote peer e31f95a2912643a5b27c656639c3c05b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:16.007368 22078 consensus_queue.cc:1035] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:16.007529 22105 consensus_queue.cc:1035] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:16.022382 21543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:16.022469 21543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:16.023206 21543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: Sending DeleteTablet for 3 replicas of tablet 756e0e161c0043d1a56dffefa0c93613
I20260806 22:42:16.023519 21973 tablet_service.cc:1514] Processing DeleteTablet for tablet 756e0e161c0043d1a56dffefa0c93613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:16 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:46834
I20260806 22:42:16.023552 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 756e0e161c0043d1a56dffefa0c93613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:16 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:58392
I20260806 22:42:16.023753 21832 tablet_service.cc:1514] Processing DeleteTablet for tablet 756e0e161c0043d1a56dffefa0c93613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:16 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:54844
I20260806 22:42:16.023984 22111 tablet_replica.cc:330] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: stopping tablet replica
I20260806 22:42:16.023979 21543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:42:16.024088 22111 raft_consensus.cc:2229] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 1 LEADER]: Raft consensus shutting down.
W20260806 22:42:16.024132 21543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:16.024191 22111 raft_consensus.cc:2258] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:16.024456 22109 tablet_replica.cc:330] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: stopping tablet replica
I20260806 22:42:16.024557 22109 raft_consensus.cc:2229] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:16.024611 22109 raft_consensus.cc:2258] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:16.024763 22110 tablet_replica.cc:330] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: stopping tablet replica
I20260806 22:42:16.025281 22110 raft_consensus.cc:2229] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:16.025332 22109 ts_tablet_manager.cc:1905] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:16.025353 22110 raft_consensus.cc:2258] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:16.025624 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 64b5a7668f8f45f6915197308bc02abe (DEFAULT_TABLE table=test-table [id=b943418bbe3d4502898017ffe581f287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:16.025741 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b5a7668f8f45f6915197308bc02abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:16.025779 22110 ts_tablet_manager.cc:1905] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:16.026731 21973 tablet_service.cc:1467] Processing CreateTablet for tablet 64b5a7668f8f45f6915197308bc02abe (DEFAULT_TABLE table=test-table [id=b943418bbe3d4502898017ffe581f287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:16.026826 21973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b5a7668f8f45f6915197308bc02abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:16.027119 22109 ts_tablet_manager.cc:1918] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:16.027184 22109 log.cc:1196] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/756e0e161c0043d1a56dffefa0c93613
I20260806 22:42:16.027431 22109 ts_tablet_manager.cc:1939] T 756e0e161c0043d1a56dffefa0c93613 P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting consensus metadata
I20260806 22:42:16.027853 21528 catalog_manager.cc:4841] TS ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635): tablet 756e0e161c0043d1a56dffefa0c93613 (table test-table [id=4a588220a74645aa9c13acc573c5fb86]) successfully deleted
I20260806 22:42:16.028117 22111 ts_tablet_manager.cc:1905] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:16.028626 21832 tablet_service.cc:1467] Processing CreateTablet for tablet 64b5a7668f8f45f6915197308bc02abe (DEFAULT_TABLE table=test-table [id=b943418bbe3d4502898017ffe581f287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:16.028728 21832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b5a7668f8f45f6915197308bc02abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:16.029785 22068 tablet_bootstrap.cc:492] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: Bootstrap starting.
I20260806 22:42:16.029896 22111 ts_tablet_manager.cc:1918] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:16.029961 22111 log.cc:1196] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/756e0e161c0043d1a56dffefa0c93613
I20260806 22:42:16.029996 22072 tablet_bootstrap.cc:492] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: Bootstrap starting.
I20260806 22:42:16.030197 22068 tablet_bootstrap.cc:654] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:16.030220 22111 ts_tablet_manager.cc:1939] T 756e0e161c0043d1a56dffefa0c93613 P e31f95a2912643a5b27c656639c3c05b: Deleting consensus metadata
I20260806 22:42:16.030424 22072 tablet_bootstrap.cc:654] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:16.030575 21529 catalog_manager.cc:4841] TS e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513): tablet 756e0e161c0043d1a56dffefa0c93613 (table test-table [id=4a588220a74645aa9c13acc573c5fb86]) successfully deleted
I20260806 22:42:16.031170 22069 tablet_bootstrap.cc:492] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: Bootstrap starting.
I20260806 22:42:16.031553 22069 tablet_bootstrap.cc:654] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:16.031826 22110 ts_tablet_manager.cc:1918] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:16.031922 22110 log.cc:1196] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/756e0e161c0043d1a56dffefa0c93613
I20260806 22:42:16.032198 22110 ts_tablet_manager.cc:1939] T 756e0e161c0043d1a56dffefa0c93613 P b748905c69b14519b5cfa9616fe3c028: Deleting consensus metadata
I20260806 22:42:16.032351 22069 tablet_bootstrap.cc:492] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: No bootstrap required, opened a new log
I20260806 22:42:16.032403 22069 ts_tablet_manager.cc:1397] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:16.032536 22069 raft_consensus.cc:348] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.032609 22069 raft_consensus.cc:374] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:16.032630 22069 raft_consensus.cc:729] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b748905c69b14519b5cfa9616fe3c028, State: Initialized, Role: FOLLOWER
I20260806 22:42:16.032656 21529 catalog_manager.cc:4841] TS b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681): tablet 756e0e161c0043d1a56dffefa0c93613 (table test-table [id=4a588220a74645aa9c13acc573c5fb86]) successfully deleted
I20260806 22:42:16.032672 22069 consensus_queue.cc:260] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.032750 22069 ts_tablet_manager.cc:1428] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:16.032943 22068 tablet_bootstrap.cc:492] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: No bootstrap required, opened a new log
I20260806 22:42:16.033008 22068 ts_tablet_manager.cc:1397] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:42:16.033143 22068 raft_consensus.cc:348] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.033188 22068 raft_consensus.cc:374] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:16.033201 22068 raft_consensus.cc:729] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd6729f42fa43298b88a1ae3ad9fd5c, State: Initialized, Role: FOLLOWER
I20260806 22:42:16.033257 22068 consensus_queue.cc:260] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.033357 22068 ts_tablet_manager.cc:1428] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:16.035036 22072 tablet_bootstrap.cc:492] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: No bootstrap required, opened a new log
I20260806 22:42:16.035108 22072 ts_tablet_manager.cc:1397] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260806 22:42:16.035265 22072 raft_consensus.cc:348] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.035349 22072 raft_consensus.cc:374] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:16.035374 22072 raft_consensus.cc:729] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31f95a2912643a5b27c656639c3c05b, State: Initialized, Role: FOLLOWER
I20260806 22:42:16.035423 22072 consensus_queue.cc:260] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.035535 22072 ts_tablet_manager.cc:1428] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
W20260806 22:42:16.090890 22045 tablet.cc:2369] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:16.255406 22074 raft_consensus.cc:482] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:16.255533 22074 raft_consensus.cc:504] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.255944 22074 leader_election.cc:288] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513), ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:16.259888 22064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.256188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:42617 (local address 127.19.74.195:39635)
0806 22:42:16.256262 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:16.256264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:16.256304 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:42:16.256387 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:16.256388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:16.256397 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:16.256452 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:16.256654 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.257409 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.258972 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.259535 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.259603 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.259619 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:16.259782 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 22:42:16.259820 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:42:16.259888 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.256164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:47807 (local address 127.19.74.194:40513)
0806 22:42:16.256215 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:16.256217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:16.256438 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:42:16.256503 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:16.256504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:16.256513 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:16.256566 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:16.256723 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.257424 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.258972 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.259527 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.259603 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.259614 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:16.259783 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:42:16.259818 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:16.259924 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.256116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:39635 (local address 127.19.74.193:42617)
0806 22:42:16.256249 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:16.256261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:16.256322 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:16.256480 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:16.256482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:16.256484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:16.256568 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.256570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.257505 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.257506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.258881 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.258884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.259617 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.259618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.259696 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.259703 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:42:16.259762 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:42:16.259818 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":54,"thread_start_us":68,"threads_started":1}
I20260806 22:42:16.259943 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.256133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:40513 (local address 127.19.74.193:47807)
0806 22:42:16.256361 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:16.256371 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:16.256447 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:16.256578 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:16.256579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:16.256581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:16.256650 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.256651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.257502 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.257503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.258881 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.258884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.259618 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.259619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.259696 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.259704 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:42:16.259762 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:42:16.259818 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":51,"thread_start_us":79,"threads_started":1}
I20260806 22:42:16.260196 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b5a7668f8f45f6915197308bc02abe" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" is_pre_election: true
I20260806 22:42:16.260290 21996 raft_consensus.cc:2454] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 0.
I20260806 22:42:16.260258 21853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b5a7668f8f45f6915197308bc02abe" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e31f95a2912643a5b27c656639c3c05b" is_pre_election: true
I20260806 22:42:16.260401 21853 raft_consensus.cc:2454] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 0.
I20260806 22:42:16.260628 21636 leader_election.cc:302] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b748905c69b14519b5cfa9616fe3c028, ecd6729f42fa43298b88a1ae3ad9fd5c; no voters:
I20260806 22:42:16.260773 22074 raft_consensus.cc:2781] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:16.260830 22074 raft_consensus.cc:482] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:16.260860 22074 raft_consensus.cc:3037] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:16.261561 22074 raft_consensus.cc:504] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.261740 22074 leader_election.cc:288] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 election: Requested vote from peers e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513), ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:16.261891 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b5a7668f8f45f6915197308bc02abe" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c"
I20260806 22:42:16.261969 21996 raft_consensus.cc:3037] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:16.262526 21996 raft_consensus.cc:2454] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 1.
I20260806 22:42:16.262666 21853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b5a7668f8f45f6915197308bc02abe" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e31f95a2912643a5b27c656639c3c05b"
I20260806 22:42:16.262696 21636 leader_election.cc:302] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b748905c69b14519b5cfa9616fe3c028, ecd6729f42fa43298b88a1ae3ad9fd5c; no voters:
I20260806 22:42:16.262763 21853 raft_consensus.cc:3037] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:16.262811 22074 raft_consensus.cc:2781] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:16.262921 22074 raft_consensus.cc:686] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 1 LEADER]: Becoming Leader. State: Replica: b748905c69b14519b5cfa9616fe3c028, State: Running, Role: LEADER
I20260806 22:42:16.262989 22074 consensus_queue.cc:237] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:16.263669 21853 raft_consensus.cc:2454] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 1.
I20260806 22:42:16.263727 21543 catalog_manager.cc:5495] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 reported cstate change: term changed from 0 to 1, leader changed from <none> to b748905c69b14519b5cfa9616fe3c028 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "b748905c69b14519b5cfa9616fe3c028" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } health_report { overall_health: UNKNOWN } } }
W20260806 22:42:16.278137 21759 tablet.cc:2369] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:16.317873 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.315748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.19.74.193:37681)
0806 22:42:16.315792 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:16.315794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:16.315801 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:16.315892 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:16.315893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:16.315912 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:16.315974 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:16.316126 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.316566 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.317208 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.317392 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.317438 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.317546 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:16.317647 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:16.317694 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:16.317731 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:42:16.317744 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:16.322827 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.320528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:56057 (local address 127.19.74.194:40513)
0806 22:42:16.320583 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:16.320585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:16.320632 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:42:16.320705 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:16.320706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:16.320716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:16.320784 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:16.320919 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.321380 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.322132 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.322321 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.322396 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.322565 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:16.322604 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:16.322679 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:16.322743 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:42:16.322760 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:42:16.322870 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.320386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:40513 (local address 127.19.74.193:56057)
0806 22:42:16.320571 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:16.320582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:16.320649 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:16.320788 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:16.320789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:16.320791 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:16.320845 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.320847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.321457 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.321459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.322043 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.322045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.322399 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.322400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.322464 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.322485 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:16.322684 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:16.322686 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:16.322737 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:42:16.322783 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260806 22:42:16.325032 21996 raft_consensus.cc:1261] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Refusing update from remote peer b748905c69b14519b5cfa9616fe3c028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:16.325057 21853 raft_consensus.cc:1261] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Refusing update from remote peer b748905c69b14519b5cfa9616fe3c028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:16.325328 22117 consensus_queue.cc:1035] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:16.325448 22074 consensus_queue.cc:1035] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:16.333470 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.331305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:60949 (local address 127.19.74.193:37681)
0806 22:42:16.331338 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:16.331340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:16.331375 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:42:16.331435 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:16.331436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:16.331445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:16.331500 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:16.331613 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.332057 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.332770 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.332975 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.333048 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.333226 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:16.333256 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:16.333319 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:16.333377 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 22:42:16.333396 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:16.333492 21907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:16.331267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:37681 (local address 127.19.74.194:60949)
0806 22:42:16.331315 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:16.331326 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:16.331388 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:16.331505 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:16.331506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:16.331507 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:16.331544 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.331545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.332145 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.332147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.332680 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:16.332682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:16.333082 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:16.333083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:16.333140 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:16.333146 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:16.333326 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:16.333327 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:42:16.333374 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:42:16.333426 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:42:16.333791 22096 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:42:17.854230 21544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:17.854341 21544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:17.855118 21544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: Sending DeleteTablet for 3 replicas of tablet 64b5a7668f8f45f6915197308bc02abe
I20260806 22:42:17.855422 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b5a7668f8f45f6915197308bc02abe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:17 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:58392
I20260806 22:42:17.855418 21973 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b5a7668f8f45f6915197308bc02abe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:17 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:46834
I20260806 22:42:17.855485 21832 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b5a7668f8f45f6915197308bc02abe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:17 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:54844
I20260806 22:42:17.855690 22153 tablet_replica.cc:330] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: stopping tablet replica
I20260806 22:42:17.855978 22153 raft_consensus.cc:2229] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:17.856127 22153 raft_consensus.cc:2258] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:17.856406 22153 ts_tablet_manager.cc:1905] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:17.856268 22152 tablet_replica.cc:330] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: stopping tablet replica
I20260806 22:42:17.857661 22152 raft_consensus.cc:2229] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:17.857707 22152 raft_consensus.cc:2258] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:17.857954 22152 ts_tablet_manager.cc:1905] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:17.858421 22153 ts_tablet_manager.cc:1918] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:42:17.858491 22153 log.cc:1196] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/64b5a7668f8f45f6915197308bc02abe
I20260806 22:42:17.858812 22153 ts_tablet_manager.cc:1939] T 64b5a7668f8f45f6915197308bc02abe P b748905c69b14519b5cfa9616fe3c028: Deleting consensus metadata
I20260806 22:42:17.859120 21529 catalog_manager.cc:4841] TS b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681): tablet 64b5a7668f8f45f6915197308bc02abe (table test-table [id=b943418bbe3d4502898017ffe581f287]) successfully deleted
I20260806 22:42:17.859750 22152 ts_tablet_manager.cc:1918] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:42:17.859800 22152 log.cc:1196] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/64b5a7668f8f45f6915197308bc02abe
I20260806 22:42:17.860010 22152 ts_tablet_manager.cc:1939] T 64b5a7668f8f45f6915197308bc02abe P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting consensus metadata
I20260806 22:42:17.860179 22154 tablet_replica.cc:330] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: stopping tablet replica
I20260806 22:42:17.860232 22154 raft_consensus.cc:2229] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:17.860285 22154 raft_consensus.cc:2258] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:17.860553 21528 catalog_manager.cc:4841] TS ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635): tablet 64b5a7668f8f45f6915197308bc02abe (table test-table [id=b943418bbe3d4502898017ffe581f287]) successfully deleted
I20260806 22:42:17.860651 22154 ts_tablet_manager.cc:1905] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:17.862040 22154 ts_tablet_manager.cc:1918] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:42:17.862142 22154 log.cc:1196] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/64b5a7668f8f45f6915197308bc02abe
I20260806 22:42:17.862349 22154 ts_tablet_manager.cc:1939] T 64b5a7668f8f45f6915197308bc02abe P e31f95a2912643a5b27c656639c3c05b: Deleting consensus metadata
I20260806 22:42:17.862673 21529 catalog_manager.cc:4841] TS e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513): tablet 64b5a7668f8f45f6915197308bc02abe (table test-table [id=b943418bbe3d4502898017ffe581f287]) successfully deleted
Aug 06 22:42:17 dist-test-slave-pgkr krb5kdc[21496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:42:17 dist-test-slave-pgkr krb5kdc[21496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:17.880599 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:17.872884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.19.74.254:46127)
0806 22:42:17.873034 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:17.873036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:17.873088 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:42:17.873200 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:17.873201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:17.873214 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:17.873293 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:17.873920 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:17.874438 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:17.875260 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:17.875486 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:17.875572 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:17.877956 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:17.877957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:17.877958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:17.877965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:17.879378 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:17.879771 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:17.879773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:17.879774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:17.879838 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:17.880056 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:17.880057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:17.880058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:17.880183 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:17.880286 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:17.880406 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:42:17.880486 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260806 22:42:17.881906 21544 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:17.882087 21544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:17.883576 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 6e980358232047a5840b62db12ec96ce (DEFAULT_TABLE table=test-table [id=46f18e663dba422d8eed03b391b554d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:17.883591 21832 tablet_service.cc:1467] Processing CreateTablet for tablet 6e980358232047a5840b62db12ec96ce (DEFAULT_TABLE table=test-table [id=46f18e663dba422d8eed03b391b554d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:17.883590 21973 tablet_service.cc:1467] Processing CreateTablet for tablet 6e980358232047a5840b62db12ec96ce (DEFAULT_TABLE table=test-table [id=46f18e663dba422d8eed03b391b554d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:17.883725 21832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e980358232047a5840b62db12ec96ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:17.883723 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e980358232047a5840b62db12ec96ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:17.883744 21973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e980358232047a5840b62db12ec96ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:17.884840 22163 tablet_bootstrap.cc:492] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: Bootstrap starting.
I20260806 22:42:17.885155 22164 tablet_bootstrap.cc:492] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: Bootstrap starting.
I20260806 22:42:17.885222 22165 tablet_bootstrap.cc:492] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: Bootstrap starting.
I20260806 22:42:17.885460 22163 tablet_bootstrap.cc:654] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:17.885537 22164 tablet_bootstrap.cc:654] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:17.885633 22165 tablet_bootstrap.cc:654] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:17.886198 22164 tablet_bootstrap.cc:492] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: No bootstrap required, opened a new log
I20260806 22:42:17.886233 22165 tablet_bootstrap.cc:492] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: No bootstrap required, opened a new log
I20260806 22:42:17.886282 22164 ts_tablet_manager.cc:1397] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:17.886298 22165 ts_tablet_manager.cc:1397] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:17.886394 22164 raft_consensus.cc:348] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.886466 22164 raft_consensus.cc:374] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:17.886420 22165 raft_consensus.cc:348] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.886482 22164 raft_consensus.cc:729] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31f95a2912643a5b27c656639c3c05b, State: Initialized, Role: FOLLOWER
I20260806 22:42:17.886487 22165 raft_consensus.cc:374] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:17.886509 22165 raft_consensus.cc:729] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b748905c69b14519b5cfa9616fe3c028, State: Initialized, Role: FOLLOWER
I20260806 22:42:17.886538 22164 consensus_queue.cc:260] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.886654 22164 ts_tablet_manager.cc:1428] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:17.886564 22165 consensus_queue.cc:260] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.886765 22165 ts_tablet_manager.cc:1428] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:17.886850 22163 tablet_bootstrap.cc:492] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: No bootstrap required, opened a new log
I20260806 22:42:17.886930 22163 ts_tablet_manager.cc:1397] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:17.887107 22163 raft_consensus.cc:348] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.887197 22163 raft_consensus.cc:374] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:17.887256 22163 raft_consensus.cc:729] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd6729f42fa43298b88a1ae3ad9fd5c, State: Initialized, Role: FOLLOWER
I20260806 22:42:17.887317 22163 consensus_queue.cc:260] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.887534 22163 ts_tablet_manager.cc:1428] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:42:17.919150 22147 raft_consensus.cc:482] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:17.919219 22147 raft_consensus.cc:504] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.919407 22147 leader_election.cc:288] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513), ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:17.919581 21853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e980358232047a5840b62db12ec96ce" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e31f95a2912643a5b27c656639c3c05b" is_pre_election: true
I20260806 22:42:17.919615 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e980358232047a5840b62db12ec96ce" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" is_pre_election: true
I20260806 22:42:17.919680 21853 raft_consensus.cc:2454] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 0.
I20260806 22:42:17.919683 21996 raft_consensus.cc:2454] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 0.
I20260806 22:42:17.919819 21636 leader_election.cc:302] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b748905c69b14519b5cfa9616fe3c028, ecd6729f42fa43298b88a1ae3ad9fd5c; no voters:
I20260806 22:42:17.919906 22147 raft_consensus.cc:2781] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:17.919943 22147 raft_consensus.cc:482] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:17.919960 22147 raft_consensus.cc:3037] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:17.920735 22147 raft_consensus.cc:504] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.920876 22147 leader_election.cc:288] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 election: Requested vote from peers e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513), ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:17.921010 21853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e980358232047a5840b62db12ec96ce" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e31f95a2912643a5b27c656639c3c05b"
I20260806 22:42:17.921048 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e980358232047a5840b62db12ec96ce" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c"
I20260806 22:42:17.921109 21996 raft_consensus.cc:3037] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:17.921114 21853 raft_consensus.cc:3037] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:17.921664 21996 raft_consensus.cc:2454] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 1.
I20260806 22:42:17.921770 21853 raft_consensus.cc:2454] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 1.
I20260806 22:42:17.921833 21636 leader_election.cc:302] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b748905c69b14519b5cfa9616fe3c028, ecd6729f42fa43298b88a1ae3ad9fd5c; no voters:
I20260806 22:42:17.921941 22147 raft_consensus.cc:2781] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:17.922003 22147 raft_consensus.cc:686] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 1 LEADER]: Becoming Leader. State: Replica: b748905c69b14519b5cfa9616fe3c028, State: Running, Role: LEADER
I20260806 22:42:17.922050 22147 consensus_queue.cc:237] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.922725 21543 catalog_manager.cc:5495] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 reported cstate change: term changed from 0 to 1, leader changed from <none> to b748905c69b14519b5cfa9616fe3c028 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "b748905c69b14519b5cfa9616fe3c028" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:17.934813 22172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:17.932608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.19.74.193:37681)
0806 22:42:17.932754 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:17.932756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:17.932762 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:42:17.932782 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:17.932782 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:17.932792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:17.932850 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:17.932977 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:17.933446 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:17.934138 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:17.934320 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:17.934369 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:17.934500 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:17.934529 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:17.934563 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:17.934683 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:42:17.934704 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260806 22:42:17.936741 21853 raft_consensus.cc:1261] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Refusing update from remote peer b748905c69b14519b5cfa9616fe3c028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:17.936748 21996 raft_consensus.cc:1261] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Refusing update from remote peer b748905c69b14519b5cfa9616fe3c028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:17.936985 22147 consensus_queue.cc:1035] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:17.937078 22143 consensus_queue.cc:1035] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:17.946274 21543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:17.946398 21543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:17.947204 21543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: Sending DeleteTablet for 3 replicas of tablet 6e980358232047a5840b62db12ec96ce
I20260806 22:42:17.947432 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e980358232047a5840b62db12ec96ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:17 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:58392
I20260806 22:42:17.947444 21973 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e980358232047a5840b62db12ec96ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:17 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:46834
I20260806 22:42:17.947489 21832 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e980358232047a5840b62db12ec96ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:17 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:54844
I20260806 22:42:17.947533 22152 tablet_replica.cc:330] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: stopping tablet replica
I20260806 22:42:17.947540 22153 tablet_replica.cc:330] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: stopping tablet replica
I20260806 22:42:17.947580 22152 raft_consensus.cc:2229] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:17.947590 22153 raft_consensus.cc:2229] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:17.947616 22152 raft_consensus.cc:2258] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:17.947641 22153 raft_consensus.cc:2258] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:17.947681 22154 tablet_replica.cc:330] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: stopping tablet replica
I20260806 22:42:17.947738 22154 raft_consensus.cc:2229] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:17.947788 22154 raft_consensus.cc:2258] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:17.947834 21543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:17.947968 21543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:17.947960 22153 ts_tablet_manager.cc:1905] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:17.948064 22152 ts_tablet_manager.cc:1905] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:17.949769 22154 ts_tablet_manager.cc:1905] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:17.949905 22152 ts_tablet_manager.cc:1918] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:17.949959 22152 log.cc:1196] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/6e980358232047a5840b62db12ec96ce
I20260806 22:42:17.949977 22153 ts_tablet_manager.cc:1918] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:17.950034 22153 log.cc:1196] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/6e980358232047a5840b62db12ec96ce
I20260806 22:42:17.950196 22152 ts_tablet_manager.cc:1939] T 6e980358232047a5840b62db12ec96ce P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting consensus metadata
I20260806 22:42:17.950300 22153 ts_tablet_manager.cc:1939] T 6e980358232047a5840b62db12ec96ce P b748905c69b14519b5cfa9616fe3c028: Deleting consensus metadata
I20260806 22:42:17.950510 21832 tablet_service.cc:1467] Processing CreateTablet for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9 (DEFAULT_TABLE table=test-table [id=4c7ae03267a7497390cef190340a9f61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:17.950657 21832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:17.951416 22154 ts_tablet_manager.cc:1918] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:17.951411 21529 catalog_manager.cc:4841] TS b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681): tablet 6e980358232047a5840b62db12ec96ce (table test-table [id=46f18e663dba422d8eed03b391b554d9]) successfully deleted
I20260806 22:42:17.951478 22154 log.cc:1196] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/6e980358232047a5840b62db12ec96ce
I20260806 22:42:17.951469 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9 (DEFAULT_TABLE table=test-table [id=4c7ae03267a7497390cef190340a9f61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:17.951570 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:17.951565 21973 tablet_service.cc:1467] Processing CreateTablet for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9 (DEFAULT_TABLE table=test-table [id=4c7ae03267a7497390cef190340a9f61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:17.951659 21973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:17.951738 22154 ts_tablet_manager.cc:1939] T 6e980358232047a5840b62db12ec96ce P e31f95a2912643a5b27c656639c3c05b: Deleting consensus metadata
I20260806 22:42:17.952050 21529 catalog_manager.cc:4841] TS e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513): tablet 6e980358232047a5840b62db12ec96ce (table test-table [id=46f18e663dba422d8eed03b391b554d9]) successfully deleted
I20260806 22:42:17.952602 22165 tablet_bootstrap.cc:492] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028: Bootstrap starting.
I20260806 22:42:17.952898 22163 tablet_bootstrap.cc:492] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: Bootstrap starting.
I20260806 22:42:17.952926 22164 tablet_bootstrap.cc:492] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: Bootstrap starting.
I20260806 22:42:17.952993 22165 tablet_bootstrap.cc:654] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:17.953246 22163 tablet_bootstrap.cc:654] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:17.953272 22164 tablet_bootstrap.cc:654] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:17.953516 21528 catalog_manager.cc:4841] TS ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635): tablet 6e980358232047a5840b62db12ec96ce (table test-table [id=46f18e663dba422d8eed03b391b554d9]) successfully deleted
I20260806 22:42:17.953555 22165 tablet_bootstrap.cc:492] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028: No bootstrap required, opened a new log
I20260806 22:42:17.953600 22165 ts_tablet_manager.cc:1397] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:17.953708 22165 raft_consensus.cc:348] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.953771 22165 raft_consensus.cc:374] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:17.953785 22165 raft_consensus.cc:729] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b748905c69b14519b5cfa9616fe3c028, State: Initialized, Role: FOLLOWER
I20260806 22:42:17.953830 22164 tablet_bootstrap.cc:492] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: No bootstrap required, opened a new log
I20260806 22:42:17.953840 22165 consensus_queue.cc:260] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.953863 22164 ts_tablet_manager.cc:1397] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:17.953950 22165 ts_tablet_manager.cc:1428] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:17.953954 22164 raft_consensus.cc:348] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.953998 22164 raft_consensus.cc:374] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:17.954011 22164 raft_consensus.cc:729] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31f95a2912643a5b27c656639c3c05b, State: Initialized, Role: FOLLOWER
I20260806 22:42:17.954077 22164 consensus_queue.cc:260] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.954180 22164 ts_tablet_manager.cc:1428] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:17.954599 22163 tablet_bootstrap.cc:492] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: No bootstrap required, opened a new log
I20260806 22:42:17.954679 22163 ts_tablet_manager.cc:1397] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:17.954818 22163 raft_consensus.cc:348] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.954880 22163 raft_consensus.cc:374] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:17.954895 22163 raft_consensus.cc:729] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd6729f42fa43298b88a1ae3ad9fd5c, State: Initialized, Role: FOLLOWER
I20260806 22:42:17.954963 22163 consensus_queue.cc:260] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:17.955058 22163 ts_tablet_manager.cc:1428] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:18.106400 22143 raft_consensus.cc:482] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:18.106514 22143 raft_consensus.cc:504] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:18.106737 22143 leader_election.cc:288] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513), ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:18.106932 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ceda7824fd342aa9b9d8c44b8a2a7c9" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" is_pre_election: true
I20260806 22:42:18.107034 21996 raft_consensus.cc:2454] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 0.
I20260806 22:42:18.106974 21853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ceda7824fd342aa9b9d8c44b8a2a7c9" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e31f95a2912643a5b27c656639c3c05b" is_pre_election: true
I20260806 22:42:18.107107 21853 raft_consensus.cc:2454] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 0.
I20260806 22:42:18.107215 21636 leader_election.cc:302] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b748905c69b14519b5cfa9616fe3c028, ecd6729f42fa43298b88a1ae3ad9fd5c; no voters:
I20260806 22:42:18.107343 22143 raft_consensus.cc:2781] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:18.107389 22143 raft_consensus.cc:482] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:18.107429 22143 raft_consensus.cc:3037] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:18.108152 22143 raft_consensus.cc:504] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:18.108294 22143 leader_election.cc:288] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 election: Requested vote from peers e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513), ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635)
I20260806 22:42:18.108441 21853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ceda7824fd342aa9b9d8c44b8a2a7c9" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e31f95a2912643a5b27c656639c3c05b"
I20260806 22:42:18.108469 21996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ceda7824fd342aa9b9d8c44b8a2a7c9" candidate_uuid: "b748905c69b14519b5cfa9616fe3c028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c"
I20260806 22:42:18.108527 21853 raft_consensus.cc:3037] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:18.108533 21996 raft_consensus.cc:3037] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:18.109267 21996 raft_consensus.cc:2454] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 1.
I20260806 22:42:18.109313 21853 raft_consensus.cc:2454] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b748905c69b14519b5cfa9616fe3c028 in term 1.
I20260806 22:42:18.109467 21636 leader_election.cc:302] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b748905c69b14519b5cfa9616fe3c028, ecd6729f42fa43298b88a1ae3ad9fd5c; no voters:
I20260806 22:42:18.109596 22143 raft_consensus.cc:2781] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:18.109674 22143 raft_consensus.cc:686] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [term 1 LEADER]: Becoming Leader. State: Replica: b748905c69b14519b5cfa9616fe3c028, State: Running, Role: LEADER
I20260806 22:42:18.109714 22143 consensus_queue.cc:237] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } }
I20260806 22:42:18.110364 21542 catalog_manager.cc:5495] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 reported cstate change: term changed from 0 to 1, leader changed from <none> to b748905c69b14519b5cfa9616fe3c028 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "b748905c69b14519b5cfa9616fe3c028" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b748905c69b14519b5cfa9616fe3c028" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:18.138022 21996 raft_consensus.cc:1261] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Refusing update from remote peer b748905c69b14519b5cfa9616fe3c028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:18.138033 21853 raft_consensus.cc:1261] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Refusing update from remote peer b748905c69b14519b5cfa9616fe3c028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:18.138327 22143 consensus_queue.cc:1035] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd6729f42fa43298b88a1ae3ad9fd5c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:18.138465 22129 consensus_queue.cc:1035] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P b748905c69b14519b5cfa9616fe3c028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e31f95a2912643a5b27c656639c3c05b" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 40513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:42:18.142666 22136 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:42:19.662968 21542 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:19.663103 21542 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:19.663985 21542 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53d2d09585bc4ae08e5be00053c9013f: Sending DeleteTablet for 3 replicas of tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9
I20260806 22:42:19.664305 21832 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:19 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:54844
I20260806 22:42:19.664299 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:19 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:58392
I20260806 22:42:19.664343 21973 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:19 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:46834
I20260806 22:42:19.664549 22210 tablet_replica.cc:330] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: stopping tablet replica
I20260806 22:42:19.664618 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 21620
I20260806 22:42:19.665235 22210 raft_consensus.cc:2229] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:19.665290 22210 raft_consensus.cc:2258] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:19.665535 22210 ts_tablet_manager.cc:1905] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:19.667522 22210 ts_tablet_manager.cc:1918] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:42:19.667578 22210 log.cc:1196] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/7ceda7824fd342aa9b9d8c44b8a2a7c9
I20260806 22:42:19.667801 22210 ts_tablet_manager.cc:1939] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P ecd6729f42fa43298b88a1ae3ad9fd5c: Deleting consensus metadata
I20260806 22:42:19.668365 21528 catalog_manager.cc:4841] TS ecd6729f42fa43298b88a1ae3ad9fd5c (127.19.74.195:39635): tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9 (table test-table [id=4c7ae03267a7497390cef190340a9f61]) successfully deleted
I20260806 22:42:19.668485 22208 tablet_replica.cc:330] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: stopping tablet replica
I20260806 22:42:19.668553 22208 raft_consensus.cc:2229] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:19.668581 22208 raft_consensus.cc:2258] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:19.668812 22208 ts_tablet_manager.cc:1905] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:19.670200 22208 ts_tablet_manager.cc:1918] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:42:19.670274 22208 log.cc:1196] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/7ceda7824fd342aa9b9d8c44b8a2a7c9
I20260806 22:42:19.670482 22208 ts_tablet_manager.cc:1939] T 7ceda7824fd342aa9b9d8c44b8a2a7c9 P e31f95a2912643a5b27c656639c3c05b: Deleting consensus metadata
I20260806 22:42:19.670863 21529 catalog_manager.cc:4841] TS e31f95a2912643a5b27c656639c3c05b (127.19.74.194:40513): tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9 (table test-table [id=4c7ae03267a7497390cef190340a9f61]) successfully deleted
W20260806 22:42:19.674255 21529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:37681 (error 108)
I20260806 22:42:19.674538 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 21767
I20260806 22:42:19.675662 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:19.674640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:37681 (local address 127.0.0.1:58422)
0806 22:42:19.675558 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:19.675613 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:37681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":64,"threads_started":1}
W20260806 22:42:19.675789 21529 catalog_manager.cc:4584] TS b748905c69b14519b5cfa9616fe3c028 (127.19.74.193:37681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ceda7824fd342aa9b9d8c44b8a2a7c9: Network error: Client connection negotiation failed: client connection to 127.19.74.193:37681: connect: Connection refused (error 111)
I20260806 22:42:19.680869 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 21910
I20260806 22:42:19.686317 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 21510
2026-08-06T22:42:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5653 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:19 dist-test-slave-pgkr krb5kdc[22215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:19 dist-test-slave-pgkr krb5kdc[22215](info): set up 2 sockets
Aug 06 22:42:19 dist-test-slave-pgkr krb5kdc[22215](info): commencing operation
krb5kdc: starting...
W20260806 22:42:20.729030 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:20 dist-test-slave-pgkr krb5kdc[22215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:20.845468 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:33671
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:33671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:20.926218 22228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:20.926409 22228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:20.926430 22228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:20.927596 22228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:20.927636 22228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:20.927650 22228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:20.927692 22228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:20.927726 22228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:20.929162 22228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:33671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:33671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:20.929430 22228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:20.929675 22228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:20.934664 22234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:20.934718 22228 server_base.cc:1034] running on GCE node
W20260806 22:42:20.934677 22233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:20.934989 22236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:20.935253 22228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:20.935535 22228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:20.936678 22228 hybrid_clock.cc:648] HybridClock initialized: now 1786056140936660 us; error 38 us; skew 500 ppm
Aug 06 22:42:20 dist-test-slave-pgkr krb5kdc[22215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056140, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:20.939539 22228 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:20.939980 22228 webserver.cc:460] Webserver started at http://127.19.74.254:35529/ using document root <none> and password file <none>
I20260806 22:42:20.940227 22228 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:20.940272 22228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:20.940414 22228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:20.941504 22228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "514d36f6a8ba4ddc8bb50df4e8706b41"
format_stamp: "Formatted at 2026-08-06 22:42:20 on dist-test-slave-pgkr"
server_key: "f856ee634c42dec23c0775adfec1d49f"
server_key_iv: "9c5d18c17e58988e748e631dc403aa81"
server_key_version: "encryptionkey@0"
I20260806 22:42:20.941869 22228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "514d36f6a8ba4ddc8bb50df4e8706b41"
format_stamp: "Formatted at 2026-08-06 22:42:20 on dist-test-slave-pgkr"
server_key: "f856ee634c42dec23c0775adfec1d49f"
server_key_iv: "9c5d18c17e58988e748e631dc403aa81"
server_key_version: "encryptionkey@0"
I20260806 22:42:20.943320 22228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:20.944033 22244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:20.944211 22228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:20.944312 22228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "514d36f6a8ba4ddc8bb50df4e8706b41"
format_stamp: "Formatted at 2026-08-06 22:42:20 on dist-test-slave-pgkr"
server_key: "f856ee634c42dec23c0775adfec1d49f"
server_key_iv: "9c5d18c17e58988e748e631dc403aa81"
server_key_version: "encryptionkey@0"
I20260806 22:42:20.944391 22228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:20.969955 22228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:20.971750 22228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:20.971913 22228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:20.975977 22228 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:33671
I20260806 22:42:20.976023 22298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:33671 every 8 connection(s)
I20260806 22:42:20.976389 22228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:20.977276 22299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:20.979560 22299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41: Bootstrap starting.
I20260806 22:42:20.980103 22299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:20.980176 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 22228
I20260806 22:42:20.980283 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:20.980453 22299 log.cc:824] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:20.980470 19755 external_mini_cluster.cc:1428] Setting key d27cc4496668f4e8162d5f87d4ebfeb5
I20260806 22:42:20.981244 22299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41: No bootstrap required, opened a new log
I20260806 22:42:20.983736 22299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514d36f6a8ba4ddc8bb50df4e8706b41" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33671 } }
I20260806 22:42:20.983906 22299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:20.983942 22299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 514d36f6a8ba4ddc8bb50df4e8706b41, State: Initialized, Role: FOLLOWER
I20260806 22:42:20.984052 22299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514d36f6a8ba4ddc8bb50df4e8706b41" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33671 } }
I20260806 22:42:20.984125 22299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:20.984153 22299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:20.984201 22299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:20.984838 22299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514d36f6a8ba4ddc8bb50df4e8706b41" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33671 } }
I20260806 22:42:20.984927 22299 leader_election.cc:302] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 514d36f6a8ba4ddc8bb50df4e8706b41; no voters:
I20260806 22:42:20.985121 22299 leader_election.cc:288] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:20.985378 22305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:20.985448 22299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:20.985528 22305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [term 1 LEADER]: Becoming Leader. State: Replica: 514d36f6a8ba4ddc8bb50df4e8706b41, State: Running, Role: LEADER
I20260806 22:42:20.985639 22305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514d36f6a8ba4ddc8bb50df4e8706b41" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33671 } }
I20260806 22:42:20.986008 22305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 514d36f6a8ba4ddc8bb50df4e8706b41. Latest consensus state: current_term: 1 leader_uuid: "514d36f6a8ba4ddc8bb50df4e8706b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514d36f6a8ba4ddc8bb50df4e8706b41" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33671 } } }
I20260806 22:42:20.986027 22306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "514d36f6a8ba4ddc8bb50df4e8706b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514d36f6a8ba4ddc8bb50df4e8706b41" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33671 } } }
I20260806 22:42:20.986152 22306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:20.986358 22305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:20.986439 22313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:20.986986 22313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:42:20 dist-test-slave-pgkr krb5kdc[22215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:20.988575 22313 catalog_manager.cc:1347] Generated new cluster ID: 288b72717fcf4c14bba44b2e18b63779
I20260806 22:42:20.988620 22313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:20.991812 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:20.980954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.19.74.254:33671)
0806 22:42:20.981148 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:20.981155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:20.981178 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:42:20.981633 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:20.981641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:20.981687 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:20.981796 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:20.982581 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:20.983493 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:20.984538 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:20.984980 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:20.985121 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:20.988273 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:20.988277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:20.988277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:20.988287 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:20.990171 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:20.990581 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:20.990583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:20.990584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:20.990710 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:20.990916 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:20.990917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:20.990917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:20.991234 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:20.991393 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:20.991464 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:42:20.991596 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260806 22:42:20.995635 22313 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:20.996270 22313 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:20.998328 22313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 514d36f6a8ba4ddc8bb50df4e8706b41: Generated new TSK 0
I20260806 22:42:20.998490 22313 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:21.028975 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:33671
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:21.111326 22326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:21.111542 22326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:21.111580 22326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:21.112776 22326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:21.112843 22326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:21.112891 22326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:21.114357 22326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:21.114641 22326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:21.114884 22326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:21.119897 22334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:21.119915 22331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:21.120003 22332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:21.120154 22326 server_base.cc:1034] running on GCE node
I20260806 22:42:21.120345 22326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:21.120632 22326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:21.121806 22326 hybrid_clock.cc:648] HybridClock initialized: now 1786056141121804 us; error 47 us; skew 500 ppm
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056141, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:21.124266 22326 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:21.124660 22326 webserver.cc:460] Webserver started at http://127.19.74.193:44897/ using document root <none> and password file <none>
I20260806 22:42:21.124846 22326 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:21.124886 22326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:21.124969 22326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:21.125936 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "38239648d96d4a84aca2de04360f2fef"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "3c9720f8a26ff0a0add90a0d7407b5c3"
server_key_iv: "584c03e0036105ae77dca3f947714c69"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.126243 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "38239648d96d4a84aca2de04360f2fef"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "3c9720f8a26ff0a0add90a0d7407b5c3"
server_key_iv: "584c03e0036105ae77dca3f947714c69"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.127564 22326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:21.128209 22341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.128387 22326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:21.128458 22326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "38239648d96d4a84aca2de04360f2fef"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "3c9720f8a26ff0a0add90a0d7407b5c3"
server_key_iv: "584c03e0036105ae77dca3f947714c69"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.128505 22326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:21.138367 22326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:21.140148 22326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:21.140283 22326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:21.140506 22326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:21.140805 22326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:21.140838 22326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.140878 22326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:21.140904 22326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.146541 22326 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:39275
I20260806 22:42:21.146598 22458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:39275 every 8 connection(s)
I20260806 22:42:21.146976 22326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056141, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:21.154040 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 22326
I20260806 22:42:21.154142 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:21.154309 19755 external_mini_cluster.cc:1428] Setting key 16bd0ad28845da8a87f320275e2d9fe9
I20260806 22:42:21.155685 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:21.147632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:33671 (local address 127.19.74.193:34865)
0806 22:42:21.147793 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:21.147827 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:21.148318 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:21.148511 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:21.148514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:21.149043 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:21.149248 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:21.149253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.149868 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.149869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:21.150653 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:21.150656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.150990 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.150991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:21.151092 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:21.151489 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:21.151497 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:21.152819 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:21.154186 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:21.154192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:21.154195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:21.154454 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:21.154762 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:21.154763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:21.154763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:21.154901 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:21.155217 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:21.155223 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:21.155332 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 22:42:21.155415 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:42:21.155527 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260806 22:42:21.155706 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:21.147650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:34865 (local address 127.19.74.254:33671)
0806 22:42:21.147699 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:21.147702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:21.147921 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:42:21.148402 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:21.148403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:21.148412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:21.148462 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:21.149322 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.149793 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.150730 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.150920 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.150972 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:21.152901 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:21.152902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:21.152902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:21.152909 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:21.154099 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:21.154585 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:21.154586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:21.154586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:21.154685 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:21.154976 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:21.154977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:21.154978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:21.155143 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:21.155222 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:21.155464 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 22:42:21.155570 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:21.156492 22459 heartbeater.cc:347] Connected to a master server at 127.19.74.254:33671
I20260806 22:42:21.156626 22459 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:21.156808 22459 heartbeater.cc:511] Master 127.19.74.254:33671 requested a full tablet report, sending...
I20260806 22:42:21.157336 22262 ts_manager.cc:194] Registered new tserver with Master: 38239648d96d4a84aca2de04360f2fef (127.19.74.193:39275)
I20260806 22:42:21.157922 22262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:34865
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:21.179567 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:33671
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:21.263223 22464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:21.263438 22464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:21.263475 22464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:21.264705 22464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:21.264773 22464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:21.264818 22464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:21.266250 22464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:21.266510 22464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:21.266785 22464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:21.271603 22472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:21.271566 22470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:21.271576 22469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:21.271636 22464 server_base.cc:1034] running on GCE node
I20260806 22:42:21.272043 22464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:21.272290 22464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:21.273442 22464 hybrid_clock.cc:648] HybridClock initialized: now 1786056141273419 us; error 37 us; skew 500 ppm
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056141, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:21.275916 22464 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:21.276300 22464 webserver.cc:460] Webserver started at http://127.19.74.194:42293/ using document root <none> and password file <none>
I20260806 22:42:21.276472 22464 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:21.276510 22464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:21.276598 22464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:21.277544 22464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "3901bbee8e9d4e8bb8fe971ae59bf939"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "5b976ffb4ddbd58b3bf988511af92e0f"
server_key_iv: "8a8e27aaafffaacf191057e70a599e31"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.277832 22464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "3901bbee8e9d4e8bb8fe971ae59bf939"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "5b976ffb4ddbd58b3bf988511af92e0f"
server_key_iv: "8a8e27aaafffaacf191057e70a599e31"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.279171 22464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:21.279831 22479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.280005 22464 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:21.280117 22464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "3901bbee8e9d4e8bb8fe971ae59bf939"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "5b976ffb4ddbd58b3bf988511af92e0f"
server_key_iv: "8a8e27aaafffaacf191057e70a599e31"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.280192 22464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:21.303994 22464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:21.305840 22464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:21.306005 22464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:21.306253 22464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:21.306556 22464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:21.306615 22464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.306728 22464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:21.306758 22464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.312489 22464 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:42759
I20260806 22:42:21.312542 22596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:42759 every 8 connection(s)
I20260806 22:42:21.312885 22464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:21.314596 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 22464
I20260806 22:42:21.314728 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:21.314921 19755 external_mini_cluster.cc:1428] Setting key 71bd45d167f1ffa111d3a27b30d30425
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056141, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:21.322211 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:21.313638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:35069 (local address 127.19.74.254:33671)
0806 22:42:21.313683 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:21.313684 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:21.313877 (+ 193us) server_negotiation.cc:407] Connection header received
0806 22:42:21.314317 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:21.314317 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:21.314328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:21.314378 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:21.315476 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.315990 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.316988 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.317208 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.317287 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:21.319406 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:21.319408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:21.319408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:21.319416 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:21.320789 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:21.321223 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:21.321224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:21.321225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:21.321289 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:21.321547 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:21.321548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:21.321548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:21.321668 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:21.321741 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:21.322026 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:42:21.322124 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:21.322372 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:21.313619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:33671 (local address 127.19.74.194:35069)
0806 22:42:21.313764 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:21.313797 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:21.314239 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:21.314430 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:21.314433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:21.315175 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:21.315375 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:21.315379 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.316090 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.316092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:21.316889 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:21.316891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.317293 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.317295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:21.317425 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:21.317928 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:21.317938 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:21.319307 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:21.320890 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:21.320897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:21.320900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:21.321143 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:21.321366 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:21.321367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:21.321367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:21.321479 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:21.321744 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:21.321751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:21.321889 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 22:42:21.321991 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:42:21.322143 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260806 22:42:21.322912 22597 heartbeater.cc:347] Connected to a master server at 127.19.74.254:33671
I20260806 22:42:21.323007 22597 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:21.323236 22597 heartbeater.cc:511] Master 127.19.74.254:33671 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
I20260806 22:42:21.323704 22262 ts_manager.cc:194] Registered new tserver with Master: 3901bbee8e9d4e8bb8fe971ae59bf939 (127.19.74.194:42759)
I20260806 22:42:21.324137 22262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:35069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:21.340822 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:33671
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:21.423337 22602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:21.423528 22602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:21.423552 22602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:21.424605 22602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:21.424638 22602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:21.424664 22602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:21.426047 22602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:21.426240 22602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:21.426414 22602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:21.431428 22611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:21.431414 22607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:21.431396 22608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:21.431571 22602 server_base.cc:1034] running on GCE node
I20260806 22:42:21.431852 22602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:21.432133 22602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:21.433274 22602 hybrid_clock.cc:648] HybridClock initialized: now 1786056141433262 us; error 29 us; skew 500 ppm
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056141, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:21.435794 22602 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:21.436280 22602 webserver.cc:460] Webserver started at http://127.19.74.195:37081/ using document root <none> and password file <none>
I20260806 22:42:21.436506 22602 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:21.436570 22602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:21.436712 22602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:21.437705 22602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "2b53e5bf5c314ee88ecf66d82ce91231"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "596b1d45a1c4ac6d447005d39607c3c3"
server_key_iv: "f145fe9aa066b725fc68d949925bea2c"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.438063 22602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "2b53e5bf5c314ee88ecf66d82ce91231"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "596b1d45a1c4ac6d447005d39607c3c3"
server_key_iv: "f145fe9aa066b725fc68d949925bea2c"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.439484 22602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:21.440172 22618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.440343 22602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:21.440449 22602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "2b53e5bf5c314ee88ecf66d82ce91231"
format_stamp: "Formatted at 2026-08-06 22:42:21 on dist-test-slave-pgkr"
server_key: "596b1d45a1c4ac6d447005d39607c3c3"
server_key_iv: "f145fe9aa066b725fc68d949925bea2c"
server_key_version: "encryptionkey@0"
I20260806 22:42:21.440518 22602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:21.466771 22602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:21.468936 22602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:21.469102 22602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:21.469344 22602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:21.469655 22602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:21.469708 22602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.469750 22602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:21.469781 22602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:21.475600 22602 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:44923
I20260806 22:42:21.475649 22734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:44923 every 8 connection(s)
I20260806 22:42:21.476007 22602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056141, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:21.484830 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:21.476823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:39567 (local address 127.19.74.254:33671)
0806 22:42:21.476906 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:21.476909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:21.477066 (+ 157us) server_negotiation.cc:407] Connection header received
0806 22:42:21.477540 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:21.477541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:21.477553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:21.477623 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:21.478505 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.479060 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.479984 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.480171 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.480221 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:21.482262 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:21.482263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:21.482263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:21.482269 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:21.483586 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:21.484044 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:21.484045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:21.484045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:21.484109 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:21.484328 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:21.484329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:21.484329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:21.484428 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:21.484478 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:21.484706 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 22:42:21.484765 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:42:21.485060 22737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:21.476776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:33671 (local address 127.19.74.195:39567)
0806 22:42:21.476949 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:21.476986 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:21.477462 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:21.477677 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:21.477681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:21.478245 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:21.478433 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:21.478438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.479150 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.479151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:21.479913 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:21.479914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:21.480242 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:21.480242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:21.480343 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:21.480851 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:21.480859 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:21.482187 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:21.483662 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:21.483668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:21.483670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:21.483881 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:21.484170 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:21.484171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:21.484171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:21.484272 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:21.484483 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:21.484488 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:21.484594 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 22:42:21.484678 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:42:21.484793 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260806 22:42:21.485697 22735 heartbeater.cc:347] Connected to a master server at 127.19.74.254:33671
I20260806 22:42:21.485780 22735 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:21.485915 22735 heartbeater.cc:511] Master 127.19.74.254:33671 requested a full tablet report, sending...
I20260806 22:42:21.486063 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 22602
I20260806 22:42:21.486207 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:21.486407 22262 ts_manager.cc:194] Registered new tserver with Master: 2b53e5bf5c314ee88ecf66d82ce91231 (127.19.74.195:44923)
I20260806 22:42:21.486423 19755 external_mini_cluster.cc:1428] Setting key 7341376f8bee86476e5a2ff9bc2de9e9
I20260806 22:42:21.486930 22262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:39567
I20260806 22:42:21.487650 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:42:21.500911 22303 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:21.499976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.19.74.254:33671)
0806 22:42:21.500062 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:21.500065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:21.500183 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:42:21.500264 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:21.500265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:21.500280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:21.500344 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:21.500860 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39592: BlockingRecv error: recv got EOF from 127.0.0.1:39592 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:42:21.501870 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 22326
I20260806 22:42:21.508647 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 22464
I20260806 22:42:21.513414 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 22602
I20260806 22:42:21.518589 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 22228
2026-08-06T22:42:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1830 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22749](info): set up 2 sockets
Aug 06 22:42:21 dist-test-slave-pgkr krb5kdc[22749](info): commencing operation
krb5kdc: starting...
W20260806 22:42:22.568655 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:22 dist-test-slave-pgkr krb5kdc[22749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:22.689090 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34651
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:35349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:34651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:22.769917 22762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:22.770113 22762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:22.770134 22762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:22.771149 22762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:22.771181 22762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:22.771194 22762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:22.771205 22762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:22.771216 22762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:22.772646 22762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:35349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:34651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:22.772835 22762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:22.772996 22762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:22.777833 22770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:22.777833 22768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:22.777844 22767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:22.777966 22762 server_base.cc:1034] running on GCE node
I20260806 22:42:22.778340 22762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:22.778584 22762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:22.779783 22762 hybrid_clock.cc:648] HybridClock initialized: now 1786056142779772 us; error 27 us; skew 500 ppm
Aug 06 22:42:22 dist-test-slave-pgkr krb5kdc[22749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056142, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:22.782225 22762 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:22.782792 22762 webserver.cc:460] Webserver started at http://127.19.74.254:40821/ using document root <none> and password file <none>
I20260806 22:42:22.783033 22762 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:22.783098 22762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:22.783252 22762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:22.784301 22762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "2d2fff20ae0e4ab29ccbd68192b4f443"
format_stamp: "Formatted at 2026-08-06 22:42:22 on dist-test-slave-pgkr"
server_key: "b19fcdeb4f0746d8f1452ff4eda57b12"
server_key_iv: "f696b0162f0715c379e1a386bcf5e62d"
server_key_version: "encryptionkey@0"
I20260806 22:42:22.784684 22762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "2d2fff20ae0e4ab29ccbd68192b4f443"
format_stamp: "Formatted at 2026-08-06 22:42:22 on dist-test-slave-pgkr"
server_key: "b19fcdeb4f0746d8f1452ff4eda57b12"
server_key_iv: "f696b0162f0715c379e1a386bcf5e62d"
server_key_version: "encryptionkey@0"
I20260806 22:42:22.786000 22762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:42:22.786758 22777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:22.786926 22762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:22.787024 22762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "2d2fff20ae0e4ab29ccbd68192b4f443"
format_stamp: "Formatted at 2026-08-06 22:42:22 on dist-test-slave-pgkr"
server_key: "b19fcdeb4f0746d8f1452ff4eda57b12"
server_key_iv: "f696b0162f0715c379e1a386bcf5e62d"
server_key_version: "encryptionkey@0"
I20260806 22:42:22.787106 22762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:22.796497 22762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:22.798203 22762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:22.798374 22762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:22.802582 22762 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:34651
I20260806 22:42:22.802660 22830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:34651 every 8 connection(s)
I20260806 22:42:22.803229 22762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:22.803889 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 22762
I20260806 22:42:22.803995 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:22.804044 22831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:22.804250 19755 external_mini_cluster.cc:1428] Setting key 9bb5e7c1652d6cf2db6f05dec78f5138
I20260806 22:42:22.807567 22831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443: Bootstrap starting.
I20260806 22:42:22.808236 22831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:22.808593 22831 log.cc:824] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:22.809363 22831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443: No bootstrap required, opened a new log
I20260806 22:42:22.812101 22831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2fff20ae0e4ab29ccbd68192b4f443" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34651 } }
Aug 06 22:42:22 dist-test-slave-pgkr krb5kdc[22749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:22.812299 22831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:22.812352 22831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2fff20ae0e4ab29ccbd68192b4f443, State: Initialized, Role: FOLLOWER
I20260806 22:42:22.812515 22831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2fff20ae0e4ab29ccbd68192b4f443" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34651 } }
I20260806 22:42:22.812603 22831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:22.812652 22831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:22.812704 22831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:22.813314 22831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2fff20ae0e4ab29ccbd68192b4f443" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34651 } }
I20260806 22:42:22.813438 22831 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2fff20ae0e4ab29ccbd68192b4f443; no voters:
I20260806 22:42:22.813649 22831 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:22.813683 22836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:22.813813 22836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [term 1 LEADER]: Becoming Leader. State: Replica: 2d2fff20ae0e4ab29ccbd68192b4f443, State: Running, Role: LEADER
I20260806 22:42:22.813948 22831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:22.813948 22836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2fff20ae0e4ab29ccbd68192b4f443" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34651 } }
I20260806 22:42:22.814479 22836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2fff20ae0e4ab29ccbd68192b4f443. Latest consensus state: current_term: 1 leader_uuid: "2d2fff20ae0e4ab29ccbd68192b4f443" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2fff20ae0e4ab29ccbd68192b4f443" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34651 } } }
I20260806 22:42:22.814670 22836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:22.815661 22837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d2fff20ae0e4ab29ccbd68192b4f443" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2fff20ae0e4ab29ccbd68192b4f443" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34651 } } }
I20260806 22:42:22.815774 22837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443 [sys.catalog]: This master's current role is: LEADER
W20260806 22:42:22.816090 22853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:22.816155 22853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:22.816246 22845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:22.816459 22845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:22.817467 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:22.804797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.19.74.254:34651)
0806 22:42:22.804996 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:22.805005 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:22.805030 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:42:22.805734 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:22.805743 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:22.805812 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:22.805942 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:22.806843 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:22.808202 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:22.809166 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:22.809742 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:22.809838 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:22.812991 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:22.812995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:22.812995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:22.813005 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:22.815290 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:22.815743 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:22.815747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:22.815747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:22.815856 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:22.816134 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:22.816135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:22.816135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:22.816449 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:22.816593 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:22.817131 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 22:42:22.817271 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260806 22:42:22.818601 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:42:22.819085 22845 catalog_manager.cc:1347] Generated new cluster ID: bc19ba92bce44dec9f48fa4194d705da
I20260806 22:42:22.819160 22845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:22.825071 22845 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:22.825605 22845 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:22.830236 22845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d2fff20ae0e4ab29ccbd68192b4f443: Generated new TSK 0
I20260806 22:42:22.830420 22845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:42:22.838539 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:22.834994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.19.74.254:34651)
0806 22:42:22.835036 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:22.835038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:22.835148 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:42:22.835223 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:22.835224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:22.835232 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:22.835291 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:22.835691 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:22.836169 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:22.837220 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:22.837409 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:22.837456 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:22.838101 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:22.838103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:22.838103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:22.838272 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:22.838287 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:22.838292 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:22.838338 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:22.838432 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:42:22.838454 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:42:22.840813 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 22762
2026-08-06T22:42:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1321 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:22 dist-test-slave-pgkr krb5kdc[22865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:22 dist-test-slave-pgkr krb5kdc[22865](info): set up 2 sockets
Aug 06 22:42:22 dist-test-slave-pgkr krb5kdc[22865](info): commencing operation
krb5kdc: starting...
W20260806 22:42:23.884557 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:23 dist-test-slave-pgkr krb5kdc[22865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:24.003470 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:45143
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:45143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:24.087613 22878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:24.087824 22878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:24.087860 22878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:24.088956 22878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:24.089020 22878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:24.089046 22878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:24.089077 22878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:24.089100 22878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:24.090471 22878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:45143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:45143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:24.090750 22878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:24.090976 22878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:24.095688 22886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:24.095686 22885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:24.095732 22888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:24.095947 22878 server_base.cc:1034] running on GCE node
I20260806 22:42:24.096165 22878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:24.096449 22878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:24.097575 22878 hybrid_clock.cc:648] HybridClock initialized: now 1786056144097563 us; error 28 us; skew 500 ppm
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:24.099990 22878 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:24.100387 22878 webserver.cc:460] Webserver started at http://127.19.74.254:36303/ using document root <none> and password file <none>
I20260806 22:42:24.100623 22878 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:24.100690 22878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:24.100867 22878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:24.101899 22878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "75db23bb0efe4b55928bb0e524000585"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "d9c3c6d3cf6ad6f7a427ffd24f739c27"
server_key_iv: "f056fd9c7f7875c54c8b54f8303add3b"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.102267 22878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "75db23bb0efe4b55928bb0e524000585"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "d9c3c6d3cf6ad6f7a427ffd24f739c27"
server_key_iv: "f056fd9c7f7875c54c8b54f8303add3b"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.103709 22878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:24.104449 22895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.104626 22878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:24.104718 22878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "75db23bb0efe4b55928bb0e524000585"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "d9c3c6d3cf6ad6f7a427ffd24f739c27"
server_key_iv: "f056fd9c7f7875c54c8b54f8303add3b"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.104801 22878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:24.118119 22878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:24.120069 22878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:24.120224 22878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:24.124171 22878 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:45143
I20260806 22:42:24.124238 22948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:45143 every 8 connection(s)
I20260806 22:42:24.124572 22878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:24.125603 22949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:24.128013 22949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585: Bootstrap starting.
I20260806 22:42:24.128014 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 22878
I20260806 22:42:24.128129 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:24.128315 19755 external_mini_cluster.cc:1428] Setting key f3e9ecf9e540fcdd8e0dd5f86559b60d
I20260806 22:42:24.128759 22949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:24.129211 22949 log.cc:824] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:24.130091 22949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585: No bootstrap required, opened a new log
I20260806 22:42:24.132560 22949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75db23bb0efe4b55928bb0e524000585" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45143 } }
I20260806 22:42:24.132732 22949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:24.132772 22949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75db23bb0efe4b55928bb0e524000585, State: Initialized, Role: FOLLOWER
I20260806 22:42:24.132894 22949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75db23bb0efe4b55928bb0e524000585" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45143 } }
I20260806 22:42:24.132982 22949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:24.133018 22949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:24.133059 22949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:24.133715 22949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75db23bb0efe4b55928bb0e524000585" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45143 } }
I20260806 22:42:24.133811 22949 leader_election.cc:302] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75db23bb0efe4b55928bb0e524000585; no voters:
I20260806 22:42:24.134030 22949 leader_election.cc:288] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:24.134052 22954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:24.134178 22954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [term 1 LEADER]: Becoming Leader. State: Replica: 75db23bb0efe4b55928bb0e524000585, State: Running, Role: LEADER
I20260806 22:42:24.134318 22949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:24.134295 22954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75db23bb0efe4b55928bb0e524000585" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45143 } }
I20260806 22:42:24.134799 22956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75db23bb0efe4b55928bb0e524000585. Latest consensus state: current_term: 1 leader_uuid: "75db23bb0efe4b55928bb0e524000585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75db23bb0efe4b55928bb0e524000585" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45143 } } }
I20260806 22:42:24.134891 22956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:24.135085 22955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75db23bb0efe4b55928bb0e524000585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75db23bb0efe4b55928bb0e524000585" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45143 } } }
I20260806 22:42:24.135196 22955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585 [sys.catalog]: This master's current role is: LEADER
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
W20260806 22:42:24.136329 22970 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:24.136386 22970 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:24.136435 22962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:24.136620 22962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:24.138253 22962 catalog_manager.cc:1347] Generated new cluster ID: e15fb9a0d86d4189a42cdcaa74b09c29
I20260806 22:42:24.138312 22962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:24.140753 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:24.129045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.19.74.254:45143)
0806 22:42:24.129259 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:24.129269 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:24.129305 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:42:24.129861 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:24.129870 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:24.129934 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:24.130078 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:24.131457 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.132341 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.133386 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.133823 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.133931 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:24.136937 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:24.136941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:24.136941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:24.136953 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:24.139131 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.139555 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.139558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.139559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.139668 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.139884 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.139885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.139886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.140156 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:24.140306 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:24.140436 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 22:42:24.140554 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260806 22:42:24.150497 22962 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:24.151017 22962 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:24.156510 22962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75db23bb0efe4b55928bb0e524000585: Generated new TSK 0
I20260806 22:42:24.156705 22962 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:24.184861 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45143
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:24.266867 22975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:24.267112 22975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:24.267153 22975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:24.268239 22975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:24.268272 22975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:24.268298 22975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:24.269766 22975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:24.269940 22975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:24.270116 22975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:24.275087 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:24.275074 22980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:24.275072 22981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:24.275338 22975 server_base.cc:1034] running on GCE node
I20260806 22:42:24.275534 22975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:24.275781 22975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:24.276925 22975 hybrid_clock.cc:648] HybridClock initialized: now 1786056144276914 us; error 29 us; skew 500 ppm
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:24.284024 22975 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:24.284475 22975 webserver.cc:460] Webserver started at http://127.19.74.193:35879/ using document root <none> and password file <none>
I20260806 22:42:24.284713 22975 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:24.284758 22975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:24.284932 22975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:24.285983 22975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "7a38ef20c8b147a4aa64202eedf02613"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "0504af18fd32f1ed55e9eef4ab37fe59"
server_key_iv: "714b9ff4a502f61717d14af9bed4d856"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.286386 22975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "7a38ef20c8b147a4aa64202eedf02613"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "0504af18fd32f1ed55e9eef4ab37fe59"
server_key_iv: "714b9ff4a502f61717d14af9bed4d856"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.287853 22975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:24.288596 22990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.288785 22975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:24.288880 22975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "7a38ef20c8b147a4aa64202eedf02613"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "0504af18fd32f1ed55e9eef4ab37fe59"
server_key_iv: "714b9ff4a502f61717d14af9bed4d856"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.288954 22975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:24.301909 22975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:24.303745 22975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:24.303911 22975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:24.304123 22975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:24.304414 22975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:24.304448 22975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.304469 22975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:24.304513 22975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.310693 22975 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:33541
I20260806 22:42:24.310704 23108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:33541 every 8 connection(s)
I20260806 22:42:24.311025 22975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:24.319965 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:24.311732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:49071 (local address 127.19.74.254:45143)
0806 22:42:24.311779 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:24.311781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:24.311964 (+ 183us) server_negotiation.cc:407] Connection header received
0806 22:42:24.312456 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:24.312457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:24.312466 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:24.312523 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:24.313425 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.313949 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.314980 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.315197 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.315258 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:24.317238 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:24.317239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:24.317239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:24.317245 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:24.318722 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.319068 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.319068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.319068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.319155 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.319365 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.319366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.319366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.319466 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:24.319516 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:24.319750 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 22:42:24.319818 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:24.319988 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:24.311679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45143 (local address 127.19.74.193:49071)
0806 22:42:24.311834 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:24.311875 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:24.312369 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:24.312597 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:24.312600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:24.313159 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:24.313343 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:24.313348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.314029 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.314029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:24.314893 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:24.314896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.315278 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.315279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:24.315374 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:24.315806 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:24.315814 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:24.317153 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:24.318788 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:24.318795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:24.318796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:24.319003 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:24.319213 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:24.319214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:24.319214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:24.319310 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:24.319520 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:24.319524 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:24.319640 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 22:42:24.319723 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:42:24.319837 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260806 22:42:24.320034 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 22975
I20260806 22:42:24.320133 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:24.320317 19755 external_mini_cluster.cc:1428] Setting key 2f2e8532d718dbc77fc3c4de811dd473
I20260806 22:42:24.320749 23109 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45143
I20260806 22:42:24.320851 23109 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:24.321040 23109 heartbeater.cc:511] Master 127.19.74.254:45143 requested a full tablet report, sending...
I20260806 22:42:24.321622 22912 ts_manager.cc:194] Registered new tserver with Master: 7a38ef20c8b147a4aa64202eedf02613 (127.19.74.193:33541)
I20260806 22:42:24.322235 22912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:49071
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:24.345937 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45143
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:24.426390 23114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:24.426752 23114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:24.426805 23114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:24.428020 23114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:24.428061 23114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:24.428086 23114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:24.429594 23114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:24.429826 23114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:24.430071 23114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:24.434940 23119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:24.434940 23120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:24.434965 23123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:24.435341 23114 server_base.cc:1034] running on GCE node
I20260806 22:42:24.435504 23114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:24.435729 23114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:24.436863 23114 hybrid_clock.cc:648] HybridClock initialized: now 1786056144436848 us; error 34 us; skew 500 ppm
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:24.439339 23114 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:24.439800 23114 webserver.cc:460] Webserver started at http://127.19.74.194:42617/ using document root <none> and password file <none>
I20260806 22:42:24.440044 23114 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:24.440117 23114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:24.440268 23114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:24.441296 23114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "62fed0f970e04ddfbaba7fcb10d1cb61"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "e4022ba2fd5feb3dbe10b82f334cb585"
server_key_iv: "fd03e61aff6e3131e259a5ce278095c3"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.441659 23114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "62fed0f970e04ddfbaba7fcb10d1cb61"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "e4022ba2fd5feb3dbe10b82f334cb585"
server_key_iv: "fd03e61aff6e3131e259a5ce278095c3"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.442982 23114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:42:24.443743 23130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.443918 23114 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:24.444027 23114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "62fed0f970e04ddfbaba7fcb10d1cb61"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "e4022ba2fd5feb3dbe10b82f334cb585"
server_key_iv: "fd03e61aff6e3131e259a5ce278095c3"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.444100 23114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:24.457782 23114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:24.459733 23114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:24.459890 23114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:24.460125 23114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:24.460458 23114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:24.460505 23114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.460539 23114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:24.460567 23114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.466426 23114 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:41155
I20260806 22:42:24.466485 23252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:41155 every 8 connection(s)
I20260806 22:42:24.466835 23114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:24.470773 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 23114
I20260806 22:42:24.470892 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:24.471117 19755 external_mini_cluster.cc:1428] Setting key ce280188d775c117943a920519669faf
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:24.477357 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:24.467576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:41349 (local address 127.19.74.254:45143)
0806 22:42:24.467624 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:24.467626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:24.467842 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:42:24.468425 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:24.468426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:24.468442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:24.468504 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:24.469460 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.469956 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.471412 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.471686 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.471795 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:24.474341 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:24.474342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:24.474342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:24.474348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:24.475748 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.476344 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.476345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.476345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.476397 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.476737 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.476738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.476738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.476842 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:24.476891 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:24.477234 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 22:42:24.477296 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:24.477705 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:24.467531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45143 (local address 127.19.74.194:41349)
0806 22:42:24.467723 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:24.467758 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:24.468297 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:24.468587 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:24.468591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:24.469166 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:24.469349 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:24.469354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.470085 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.470086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:24.471277 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:24.471280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.471808 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.471809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:24.471944 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:24.472510 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:24.472518 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:24.474243 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:24.475840 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:24.475848 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:24.475850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:24.476239 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:24.476464 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:24.476465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:24.476465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:24.476657 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:24.476912 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:24.476920 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:24.477083 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 22:42:24.477221 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 22:42:24.477402 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260806 22:42:24.478235 23253 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45143
I20260806 22:42:24.478353 23253 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:24.478575 23253 heartbeater.cc:511] Master 127.19.74.254:45143 requested a full tablet report, sending...
I20260806 22:42:24.478979 22912 ts_manager.cc:194] Registered new tserver with Master: 62fed0f970e04ddfbaba7fcb10d1cb61 (127.19.74.194:41155)
I20260806 22:42:24.479353 22912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:41349
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:24.497833 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45143
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:24.578579 23258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:24.578845 23258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:24.578881 23258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:24.580052 23258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:24.580101 23258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:24.580163 23258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:24.581642 23258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:24.581897 23258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:24.582139 23258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:24.587112 23263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:24.587143 23264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:24.587239 23266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:24.587507 23258 server_base.cc:1034] running on GCE node
I20260806 22:42:24.587721 23258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:24.587970 23258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:24.589141 23258 hybrid_clock.cc:648] HybridClock initialized: now 1786056144589136 us; error 35 us; skew 500 ppm
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:24.591929 23258 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:24.592406 23258 webserver.cc:460] Webserver started at http://127.19.74.195:34253/ using document root <none> and password file <none>
I20260806 22:42:24.592617 23258 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:24.592675 23258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:24.592825 23258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:24.593888 23258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "54479638e3a54dc4bb23ca308f9542a2"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "f20d30b733f72ce26c1cbcc69bc82ca0"
server_key_iv: "55d9063994a5ebbd3444d956fb98e7a4"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.594261 23258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "54479638e3a54dc4bb23ca308f9542a2"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "f20d30b733f72ce26c1cbcc69bc82ca0"
server_key_iv: "55d9063994a5ebbd3444d956fb98e7a4"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.595688 23258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:24.596447 23273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.596632 23258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:24.596714 23258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "54479638e3a54dc4bb23ca308f9542a2"
format_stamp: "Formatted at 2026-08-06 22:42:24 on dist-test-slave-pgkr"
server_key: "f20d30b733f72ce26c1cbcc69bc82ca0"
server_key_iv: "55d9063994a5ebbd3444d956fb98e7a4"
server_key_version: "encryptionkey@0"
I20260806 22:42:24.596764 23258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:24.611886 23258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:24.614157 23258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:24.614269 23258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:24.614444 23258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:24.614785 23258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:24.614822 23258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.614843 23258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:24.614856 23258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:24.620644 23258 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:41543
I20260806 22:42:24.620707 23393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:41543 every 8 connection(s)
I20260806 22:42:24.621034 23258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:24.623189 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 23258
I20260806 22:42:24.623301 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:24.623515 19755 external_mini_cluster.cc:1428] Setting key d8271a9d19dd06c8463696ecb1e2068a
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:24.630326 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:24.621793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:53311 (local address 127.19.74.254:45143)
0806 22:42:24.621837 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:24.621839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:24.622096 (+ 257us) server_negotiation.cc:407] Connection header received
0806 22:42:24.622670 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:24.622671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:24.622684 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:24.622742 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:24.623798 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.624308 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.625303 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.625494 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.625583 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:24.628024 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:24.628025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:24.628026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:24.628032 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:24.629150 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.629528 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.629529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.629529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.629578 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.629795 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.629796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.629796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.629892 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:24.629941 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:24.630205 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 22:42:24.630264 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:24.630537 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:24.621788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45143 (local address 127.19.74.195:53311)
0806 22:42:24.621969 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:24.622004 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:24.622520 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:24.622814 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:24.622818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:24.623466 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:24.623705 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:24.623710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.624405 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.624406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:24.625196 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:24.625199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.625625 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.625626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:24.625736 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:24.626247 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:24.626255 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:24.627911 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:24.629228 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:24.629234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:24.629236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:24.629462 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:24.629649 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:24.629650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:24.629650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:24.629745 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:24.629962 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:24.629972 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:24.630085 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 22:42:24.630180 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:42:24.630295 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260806 22:42:24.630975 23394 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45143
I20260806 22:42:24.631090 23394 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:24.631286 23394 heartbeater.cc:511] Master 127.19.74.254:45143 requested a full tablet report, sending...
I20260806 22:42:24.631675 22912 ts_manager.cc:194] Registered new tserver with Master: 54479638e3a54dc4bb23ca308f9542a2 (127.19.74.195:41543)
I20260806 22:42:24.632059 22912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:53311
I20260806 22:42:24.635283 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[22865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056144, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:24.659507 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:24.652184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41258 (local address 127.19.74.254:45143)
0806 22:42:24.652254 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:24.652255 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:24.652388 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:42:24.652438 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:24.652439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:24.652448 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:24.652506 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:24.653055 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.653542 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.654208 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:24.654377 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:24.654432 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:24.656860 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:24.656862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:24.656863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:24.656870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:24.658516 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.658896 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.658897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.658897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.658947 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:24.659066 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:24.659067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:24.659067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:24.659200 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:24.659271 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:24.659379 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:42:24.659440 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 22:42:24.659785 22912 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41258
I20260806 22:42:24.660255 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 22975
I20260806 22:42:24.670071 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 23114
I20260806 22:42:24.678913 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 23258
I20260806 22:42:24.684517 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 22878
2026-08-06T22:42:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1844 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[23406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[23406](info): set up 2 sockets
Aug 06 22:42:24 dist-test-slave-pgkr krb5kdc[23406](info): commencing operation
krb5kdc: starting...
W20260806 22:42:25.728251 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:25 dist-test-slave-pgkr krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:25.843632 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38347
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:25.926374 23420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:25.926617 23420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:25.926678 23420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:25.927764 23420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:25.927825 23420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:25.927845 23420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:25.927865 23420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:25.927891 23420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:25.929396 23420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:25.929700 23420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:25.929921 23420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:25.934989 23425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:25.935122 23420 server_base.cc:1034] running on GCE node
W20260806 22:42:25.935019 23428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:25.934998 23426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:25.935447 23420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:25.935739 23420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:25.936877 23420 hybrid_clock.cc:648] HybridClock initialized: now 1786056145936864 us; error 27 us; skew 500 ppm
Aug 06 22:42:25 dist-test-slave-pgkr krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056145, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:25.939276 23420 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:25.939738 23420 webserver.cc:460] Webserver started at http://127.19.74.254:46379/ using document root <none> and password file <none>
I20260806 22:42:25.940001 23420 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:25.940071 23420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:25.940231 23420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:25.941289 23420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "e5b1cf2e008640a8ba5349ee5006ecb3"
format_stamp: "Formatted at 2026-08-06 22:42:25 on dist-test-slave-pgkr"
server_key: "d0165e61d278a10857132a029311ed2c"
server_key_iv: "503306a7918f85a9c01a79be161b89be"
server_key_version: "encryptionkey@0"
I20260806 22:42:25.941653 23420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "e5b1cf2e008640a8ba5349ee5006ecb3"
format_stamp: "Formatted at 2026-08-06 22:42:25 on dist-test-slave-pgkr"
server_key: "d0165e61d278a10857132a029311ed2c"
server_key_iv: "503306a7918f85a9c01a79be161b89be"
server_key_version: "encryptionkey@0"
I20260806 22:42:25.943032 23420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:25.943776 23436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:25.943948 23420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:25.944057 23420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "e5b1cf2e008640a8ba5349ee5006ecb3"
format_stamp: "Formatted at 2026-08-06 22:42:25 on dist-test-slave-pgkr"
server_key: "d0165e61d278a10857132a029311ed2c"
server_key_iv: "503306a7918f85a9c01a79be161b89be"
server_key_version: "encryptionkey@0"
I20260806 22:42:25.944140 23420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:25.958407 23420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:25.960248 23420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:25.960415 23420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:25.964465 23420 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:38347
I20260806 22:42:25.964519 23489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:38347 every 8 connection(s)
I20260806 22:42:25.965021 23420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:25.965720 23490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:25.967904 23490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3: Bootstrap starting.
I20260806 22:42:25.968446 23490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:25.968495 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 23420
I20260806 22:42:25.968622 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:25.968783 19755 external_mini_cluster.cc:1428] Setting key fa3c744bf8528b227d390028b93bc706
I20260806 22:42:25.968782 23490 log.cc:824] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:25.969558 23490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3: No bootstrap required, opened a new log
I20260806 22:42:25.971658 23490 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b1cf2e008640a8ba5349ee5006ecb3" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38347 } }
I20260806 22:42:25.971805 23490 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:25.971825 23490 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b1cf2e008640a8ba5349ee5006ecb3, State: Initialized, Role: FOLLOWER
I20260806 22:42:25.972014 23490 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b1cf2e008640a8ba5349ee5006ecb3" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38347 } }
I20260806 22:42:25.972118 23490 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:25.972164 23490 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:25.972218 23490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:25.972765 23490 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b1cf2e008640a8ba5349ee5006ecb3" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38347 } }
I20260806 22:42:25.972888 23490 leader_election.cc:302] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5b1cf2e008640a8ba5349ee5006ecb3; no voters:
I20260806 22:42:25.973070 23490 leader_election.cc:288] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:25.973120 23496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:25.973294 23496 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [term 1 LEADER]: Becoming Leader. State: Replica: e5b1cf2e008640a8ba5349ee5006ecb3, State: Running, Role: LEADER
I20260806 22:42:25.973337 23490 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:25.974772 23496 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b1cf2e008640a8ba5349ee5006ecb3" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38347 } }
Aug 06 22:42:25 dist-test-slave-pgkr krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:25.975490 23498 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5b1cf2e008640a8ba5349ee5006ecb3. Latest consensus state: current_term: 1 leader_uuid: "e5b1cf2e008640a8ba5349ee5006ecb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b1cf2e008640a8ba5349ee5006ecb3" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38347 } } }
I20260806 22:42:25.975576 23498 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [sys.catalog]: This master's current role is: LEADER
W20260806 22:42:25.975890 23512 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:25.975955 23512 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:25.976227 23497 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5b1cf2e008640a8ba5349ee5006ecb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b1cf2e008640a8ba5349ee5006ecb3" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38347 } } }
I20260806 22:42:25.976315 23497 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:25.976432 23514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:25.976641 23514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:25.978227 23514 catalog_manager.cc:1347] Generated new cluster ID: b8312264505e420bb363f1409cdc3d82
I20260806 22:42:25.978279 23514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:25.979591 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:25.969298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.19.74.254:38347)
0806 22:42:25.969504 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:25.969514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:25.969551 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:42:25.970007 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:25.970013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:25.970060 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:25.970227 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:25.970923 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:25.971730 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:25.972663 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:25.973248 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:25.973353 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:25.976224 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:25.976228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:25.976228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:25.976237 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:25.977963 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:25.978399 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:25.978402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:25.978402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:25.978500 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:25.978755 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:25.978756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:25.978756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:25.979035 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:25.979169 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:25.979293 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 22:42:25.979417 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260806 22:42:26.000392 23514 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:26.000938 23514 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:26.003436 23514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5b1cf2e008640a8ba5349ee5006ecb3: Generated new TSK 0
I20260806 22:42:26.003639 23514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:26.037782 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38347
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:26.146903 23519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:26.147090 23519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:26.147113 23519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:26.148167 23519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:26.148200 23519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:26.148236 23519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:26.149583 23519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:26.149789 23519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:26.149963 23519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:26.154915 23525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.154924 23524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.154932 23527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:26.155269 23519 server_base.cc:1034] running on GCE node
I20260806 22:42:26.155519 23519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:26.155766 23519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:26.156908 23519 hybrid_clock.cc:648] HybridClock initialized: now 1786056146156885 us; error 33 us; skew 500 ppm
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:26.159482 23519 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:26.159895 23519 webserver.cc:460] Webserver started at http://127.19.74.193:41463/ using document root <none> and password file <none>
I20260806 22:42:26.160153 23519 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:26.160224 23519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:26.160374 23519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:26.161371 23519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "8e6fb956ab514fcc943884546ebe5267"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "65884b2374795fa933d2d480c31e692a"
server_key_iv: "a42a6a614060fd7370f6834e5426dcc4"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.161758 23519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "8e6fb956ab514fcc943884546ebe5267"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "65884b2374795fa933d2d480c31e692a"
server_key_iv: "a42a6a614060fd7370f6834e5426dcc4"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.163187 23519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:26.163937 23534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.164108 23519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:26.164213 23519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "8e6fb956ab514fcc943884546ebe5267"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "65884b2374795fa933d2d480c31e692a"
server_key_iv: "a42a6a614060fd7370f6834e5426dcc4"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.164284 23519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:26.173259 23519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:26.175628 23519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:26.175735 23519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:26.175899 23519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:26.176137 23519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:26.176167 23519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.176187 23519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:26.176200 23519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.182219 23519 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:37163
I20260806 22:42:26.182289 23651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:37163 every 8 connection(s)
I20260806 22:42:26.182585 23519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:26.184137 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 23519
I20260806 22:42:26.184231 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:26.184432 19755 external_mini_cluster.cc:1428] Setting key 4fa261095e53758319f8feaae9344300
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:26.192996 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.183399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:39615 (local address 127.19.74.254:38347)
0806 22:42:26.183466 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:26.183468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:26.183666 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:42:26.184754 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:26.184756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:26.184769 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:26.184842 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:26.185559 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.186061 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.187416 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.187636 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.187689 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.189964 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:26.189965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:26.189966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:26.189974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:26.191474 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.192006 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.192007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.192008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.192061 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.192362 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.192363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.192363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.192463 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:26.192515 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:26.192850 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 22:42:26.192929 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:42:26.193269 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.183370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38347 (local address 127.19.74.193:39615)
0806 22:42:26.183526 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:26.183566 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:26.184644 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:26.184912 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:26.184916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:26.185273 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:26.185459 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.185464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.186166 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.186167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.187315 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.187318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.187720 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.187721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.187873 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.188323 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:26.188334 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:26.189871 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:26.191557 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.191566 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.191569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.191920 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.192128 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.192129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.192130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.192291 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.192530 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:26.192538 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:26.192706 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 22:42:26.192831 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 22:42:26.193010 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260806 22:42:26.193799 23652 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38347
I20260806 22:42:26.193888 23652 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:26.194078 23652 heartbeater.cc:511] Master 127.19.74.254:38347 requested a full tablet report, sending...
I20260806 22:42:26.194715 23453 ts_manager.cc:194] Registered new tserver with Master: 8e6fb956ab514fcc943884546ebe5267 (127.19.74.193:37163)
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
I20260806 22:42:26.195312 23453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:39615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:26.211382 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38347
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:26.294399 23657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:26.294674 23657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:26.294718 23657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:26.295886 23657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:26.295956 23657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:26.295996 23657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:26.297451 23657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:26.297720 23657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:26.297943 23657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:26.302942 23665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.302942 23662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.302953 23663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:26.303445 23657 server_base.cc:1034] running on GCE node
I20260806 22:42:26.303612 23657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:26.303803 23657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:26.304970 23657 hybrid_clock.cc:648] HybridClock initialized: now 1786056146304955 us; error 27 us; skew 500 ppm
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:26.307482 23657 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:26.307894 23657 webserver.cc:460] Webserver started at http://127.19.74.194:37281/ using document root <none> and password file <none>
I20260806 22:42:26.308061 23657 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:26.308104 23657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:26.308184 23657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:26.309104 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "e6b3673e17b444c481354485a59f9cd3"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "d6ed42eb2fdb377ab4acb6b18688d361"
server_key_iv: "6b63763469dde78708d67645225594d3"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.309391 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "e6b3673e17b444c481354485a59f9cd3"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "d6ed42eb2fdb377ab4acb6b18688d361"
server_key_iv: "6b63763469dde78708d67645225594d3"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.310735 23657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:26.311434 23672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.311601 23657 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:26.311709 23657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "e6b3673e17b444c481354485a59f9cd3"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "d6ed42eb2fdb377ab4acb6b18688d361"
server_key_iv: "6b63763469dde78708d67645225594d3"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.311781 23657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:26.323501 23657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:26.325227 23657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:26.325405 23657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:26.325642 23657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:26.325992 23657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:26.326064 23657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.326114 23657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:26.326143 23657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.332034 23657 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:36149
I20260806 22:42:26.332086 23787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:36149 every 8 connection(s)
I20260806 22:42:26.332453 23657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:26.336252 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 23657
I20260806 22:42:26.336385 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:26.336586 19755 external_mini_cluster.cc:1428] Setting key fcc768c105f11d509e869c9baca2f94b
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:26.342229 23791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.333162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38347 (local address 127.19.74.194:56887)
0806 22:42:26.333355 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:26.333391 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:26.334334 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:26.334637 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:26.334641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:26.334905 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:26.335107 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.335111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.335827 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.335829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.336821 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.336824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.337312 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.337314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.337434 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.337928 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:26.337936 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:26.339326 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:26.340941 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.340947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.340949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.341167 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.341373 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.341373 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.341373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.341487 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.341777 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:26.341781 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:26.341883 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 22:42:26.341961 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:42:26.342070 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260806 22:42:26.342237 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.333191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:56887 (local address 127.19.74.254:38347)
0806 22:42:26.333263 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:26.333265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:26.333486 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:42:26.334454 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:26.334456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:26.334468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:26.334530 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:26.335216 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.335727 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.336908 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.337225 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.337296 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.339422 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:26.339423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:26.339424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:26.339432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:26.340871 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.341237 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.341238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.341239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.341304 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.341551 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.341552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.341552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.341712 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:26.341780 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:26.342005 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:42:26.342094 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:42:26.342723 23788 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38347
I20260806 22:42:26.342844 23788 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:26.343027 23788 heartbeater.cc:511] Master 127.19.74.254:38347 requested a full tablet report, sending...
I20260806 22:42:26.343482 23453 ts_manager.cc:194] Registered new tserver with Master: e6b3673e17b444c481354485a59f9cd3 (127.19.74.194:36149)
I20260806 22:42:26.344081 23453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:56887
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:26.363291 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38347
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:26.442647 23794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:26.442854 23794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:26.442878 23794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:26.444023 23794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:26.444077 23794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:26.444103 23794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:26.445583 23794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:26.445822 23794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:26.446050 23794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:26.450961 23799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.450958 23802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.450963 23800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:26.451220 23794 server_base.cc:1034] running on GCE node
I20260806 22:42:26.451448 23794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:26.451695 23794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:26.452855 23794 hybrid_clock.cc:648] HybridClock initialized: now 1786056146452831 us; error 40 us; skew 500 ppm
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:26.455248 23794 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:26.455672 23794 webserver.cc:460] Webserver started at http://127.19.74.195:33063/ using document root <none> and password file <none>
I20260806 22:42:26.455911 23794 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:26.455978 23794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:26.456133 23794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:26.457123 23794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "44e663f9db824f999814b68c02604a7e"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "7558f620282cc90282eef90718274db6"
server_key_iv: "fe8d7ae9e97540cb6e59d0463f87849b"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.457535 23794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "44e663f9db824f999814b68c02604a7e"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "7558f620282cc90282eef90718274db6"
server_key_iv: "fe8d7ae9e97540cb6e59d0463f87849b"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.458850 23794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:42:26.459529 23810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.459688 23794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:26.459777 23794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "44e663f9db824f999814b68c02604a7e"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-pgkr"
server_key: "7558f620282cc90282eef90718274db6"
server_key_iv: "fe8d7ae9e97540cb6e59d0463f87849b"
server_key_version: "encryptionkey@0"
I20260806 22:42:26.459854 23794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:26.473556 23794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:26.475558 23794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:26.475718 23794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:26.475960 23794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:26.476267 23794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:26.476322 23794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.476363 23794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:26.476389 23794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.482031 23794 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:33573
I20260806 22:42:26.482100 23926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:33573 every 8 connection(s)
I20260806 22:42:26.482406 23794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:26.487939 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 23794
I20260806 22:42:26.488034 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:26.488222 19755 external_mini_cluster.cc:1428] Setting key 5f72dc0a0206e328a8c4d32d320d679c
I20260806 22:42:26.490950 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.483090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:60573 (local address 127.19.74.254:38347)
0806 22:42:26.483145 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:26.483146 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:26.483345 (+ 199us) server_negotiation.cc:407] Connection header received
0806 22:42:26.484145 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:26.484146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:26.484155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:26.484206 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:26.484767 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.485250 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.486167 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.486340 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.486390 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.488331 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:26.488333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:26.488333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:26.488348 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:26.489785 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.490152 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.490153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.490153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.490205 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.490408 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.490409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.490409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.490505 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:26.490553 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:26.490828 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 22:42:26.490889 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:42:26.491178 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.483068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38347 (local address 127.19.74.195:60573)
0806 22:42:26.483207 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:26.483243 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:26.484063 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:26.484255 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:26.484257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:26.484514 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:26.484696 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.484700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.485328 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.485329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.486087 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.486088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.486412 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.486412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.486512 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.486950 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:26.486965 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:26.488225 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:26.489854 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.489860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.489862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.490072 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.490262 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.490263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.490263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.490355 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.490557 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:26.490561 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:26.490731 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:42:26.490813 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:42:26.490927 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260806 22:42:26.491621 23927 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38347
I20260806 22:42:26.491729 23927 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:26.491899 23927 heartbeater.cc:511] Master 127.19.74.254:38347 requested a full tablet report, sending...
I20260806 22:42:26.492295 23453 ts_manager.cc:194] Registered new tserver with Master: 44e663f9db824f999814b68c02604a7e (127.19.74.195:33573)
I20260806 22:42:26.492690 23453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:60573
I20260806 22:42:26.500092 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:26.524736 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.517062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.19.74.193:37163)
0806 22:42:26.517237 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:26.517249 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:26.517261 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:42:26.517290 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:26.517292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:26.517305 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:26.517379 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:26.518154 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.518961 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.519712 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.520030 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.520082 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.522284 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:26.522292 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:26.522292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:26.522299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:26.523750 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.524058 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.524061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.524061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.524135 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.524307 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.524307 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.524308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.524426 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:26.524532 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:26.524605 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 22:42:26.524661 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:26.532845 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.526272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.19.74.254:38347)
0806 22:42:26.526334 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:26.526336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:26.526346 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:26.526419 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:26.526420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:26.526430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:26.526481 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:26.527005 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.527463 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.528158 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.528352 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.528423 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.530391 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:26.530392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:26.530393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:26.530400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:26.531795 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.532117 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.532118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.532119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.532168 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.532409 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.532410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.532410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.532547 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:26.532629 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:26.532724 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:42:26.532782 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 22:42:26.533068 23453 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48218
I20260806 22:42:26.533386 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 23519
I20260806 22:42:26.540172 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 23657
I20260806 22:42:26.545830 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 23794
I20260806 22:42:26.551218 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 23420
2026-08-06T22:42:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1866 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23939](info): set up 2 sockets
Aug 06 22:42:26 dist-test-slave-pgkr krb5kdc[23939](info): commencing operation
krb5kdc: starting...
W20260806 22:42:27.588464 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:27 dist-test-slave-pgkr krb5kdc[23939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:27.708653 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38953
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:27.791198 23952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:27.791420 23952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:27.791441 23952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:27.792594 23952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:27.792636 23952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:27.792651 23952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:27.792688 23952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:27.792716 23952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:27.794299 23952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:27.794553 23952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:27.794792 23952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:27.799479 23961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.799467 23958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.799468 23957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:27.799685 23952 server_base.cc:1034] running on GCE node
I20260806 22:42:27.799894 23952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:27.800222 23952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:27.801358 23952 hybrid_clock.cc:648] HybridClock initialized: now 1786056147801348 us; error 26 us; skew 500 ppm
Aug 06 22:42:27 dist-test-slave-pgkr krb5kdc[23939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:27.803930 23952 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:27.804291 23952 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:27.804355 23952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:27.804508 23952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:27.805605 23952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "0103fa8aa81e453180fa750fd104f099"
format_stamp: "Formatted at 2026-08-06 22:42:27 on dist-test-slave-pgkr"
server_key: "5e60a819bba3611c1b9085cb4f5397b1"
server_key_iv: "22dc0f52f9b73bf6c8d7a0e01e60a70c"
server_key_version: "encryptionkey@0"
I20260806 22:42:27.806000 23952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "0103fa8aa81e453180fa750fd104f099"
format_stamp: "Formatted at 2026-08-06 22:42:27 on dist-test-slave-pgkr"
server_key: "5e60a819bba3611c1b9085cb4f5397b1"
server_key_iv: "22dc0f52f9b73bf6c8d7a0e01e60a70c"
server_key_version: "encryptionkey@0"
I20260806 22:42:27.807389 23952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:27.808080 23967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.808246 23952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.808346 23952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "0103fa8aa81e453180fa750fd104f099"
format_stamp: "Formatted at 2026-08-06 22:42:27 on dist-test-slave-pgkr"
server_key: "5e60a819bba3611c1b9085cb4f5397b1"
server_key_iv: "22dc0f52f9b73bf6c8d7a0e01e60a70c"
server_key_version: "encryptionkey@0"
I20260806 22:42:27.808419 23952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:27.826517 23952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:27.829010 23952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:27.829190 23952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:27.833411 23952 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:38953
I20260806 22:42:27.833441 24020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:38953 every 8 connection(s)
I20260806 22:42:27.833745 23952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:27.834877 24021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:27.837201 24021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099: Bootstrap starting.
I20260806 22:42:27.837842 24021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:27.838164 24021 log.cc:824] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:27.838897 24021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099: No bootstrap required, opened a new log
I20260806 22:42:27.840443 24021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0103fa8aa81e453180fa750fd104f099" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38953 } }
I20260806 22:42:27.840576 24021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:27.840598 24021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0103fa8aa81e453180fa750fd104f099, State: Initialized, Role: FOLLOWER
I20260806 22:42:27.840739 24021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0103fa8aa81e453180fa750fd104f099" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38953 } }
I20260806 22:42:27.840811 24021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:27.840845 24021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:27.840898 24021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:27.841698 24021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0103fa8aa81e453180fa750fd104f099" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38953 } }
I20260806 22:42:27.841794 24021 leader_election.cc:302] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0103fa8aa81e453180fa750fd104f099; no voters:
I20260806 22:42:27.841982 24021 leader_election.cc:288] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:27.842053 24024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:27.842247 24024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [term 1 LEADER]: Becoming Leader. State: Replica: 0103fa8aa81e453180fa750fd104f099, State: Running, Role: LEADER
I20260806 22:42:27.842285 24021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:27.842433 24024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0103fa8aa81e453180fa750fd104f099" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38953 } }
I20260806 22:42:27.842895 24026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0103fa8aa81e453180fa750fd104f099. Latest consensus state: current_term: 1 leader_uuid: "0103fa8aa81e453180fa750fd104f099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0103fa8aa81e453180fa750fd104f099" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38953 } } }
I20260806 22:42:27.842978 24026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:27.843212 24032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:27.843245 24025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0103fa8aa81e453180fa750fd104f099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0103fa8aa81e453180fa750fd104f099" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38953 } } }
I20260806 22:42:27.843564 24025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:27.843641 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 23952
I20260806 22:42:27.843817 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:27.843981 19755 external_mini_cluster.cc:1428] Setting key 744a823391894b3631baafe16579bd9b
I20260806 22:42:27.844425 24032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:27.846066 24032 catalog_manager.cc:1347] Generated new cluster ID: 14e4216dff9a46b8b3be8d2adb28e745
I20260806 22:42:27.846127 24032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:42:27 dist-test-slave-pgkr krb5kdc[23939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:27.854103 24032 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:27.854605 24032 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:27.856499 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.844458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.19.74.254:38953)
0806 22:42:27.844625 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:27.844634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:27.845276 (+ 642us) server_negotiation.cc:407] Connection header received
0806 22:42:27.845433 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:27.845441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:27.845516 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:27.845662 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:27.846539 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.847990 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.849045 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.849678 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.849775 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.852661 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:27.852666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:27.852667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:27.852679 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:27.854851 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.855268 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.855271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.855271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.855376 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.855604 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.855605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.855606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.855883 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:27.856045 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:27.856113 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:42:27.856259 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260806 22:42:27.859540 24032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099: Generated new TSK 0
I20260806 22:42:27.859720 24032 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:27.888819 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38953
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:27.971208 24048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:27.971400 24048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:27.971422 24048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:27.972604 24048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:27.972643 24048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:27.972669 24048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:27.974254 24048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:27.974480 24048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:27.974750 24048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:27.979741 24056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.979734 24054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.979733 24053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:27.979923 24048 server_base.cc:1034] running on GCE node
I20260806 22:42:27.980178 24048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:27.980432 24048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:27.981621 24048 hybrid_clock.cc:648] HybridClock initialized: now 1786056147981600 us; error 45 us; skew 500 ppm
Aug 06 22:42:27 dist-test-slave-pgkr krb5kdc[23939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:27.984120 24048 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:27.984472 24048 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:27.984583 24048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:27.984735 24048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:27.985764 24048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "e97cab8852fb4412940f45c683bca2cb"
format_stamp: "Formatted at 2026-08-06 22:42:27 on dist-test-slave-pgkr"
server_key: "e8b0750ed31617604afea37f3d8c0eb9"
server_key_iv: "28825f1533f2fbf14b2e951af2b05365"
server_key_version: "encryptionkey@0"
I20260806 22:42:27.986130 24048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "e97cab8852fb4412940f45c683bca2cb"
format_stamp: "Formatted at 2026-08-06 22:42:27 on dist-test-slave-pgkr"
server_key: "e8b0750ed31617604afea37f3d8c0eb9"
server_key_iv: "28825f1533f2fbf14b2e951af2b05365"
server_key_version: "encryptionkey@0"
I20260806 22:42:27.987556 24048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:27.988287 24063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.988459 24048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:27.988585 24048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "e97cab8852fb4412940f45c683bca2cb"
format_stamp: "Formatted at 2026-08-06 22:42:27 on dist-test-slave-pgkr"
server_key: "e8b0750ed31617604afea37f3d8c0eb9"
server_key_iv: "28825f1533f2fbf14b2e951af2b05365"
server_key_version: "encryptionkey@0"
I20260806 22:42:27.988659 24048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:28.005086 24048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:28.008077 24048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:28.008247 24048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:28.008549 24048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:28.008910 24048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:28.008965 24048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.009037 24048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:28.009068 24048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.016774 24048 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:44599
I20260806 22:42:28.016808 24182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:44599 every 8 connection(s)
I20260806 22:42:28.017114 24048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:28 dist-test-slave-pgkr krb5kdc[23939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:28.024245 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 24048
I20260806 22:42:28.024369 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:28.024607 19755 external_mini_cluster.cc:1428] Setting key c29a5f24f93c3d4a60d4895517a62493
I20260806 22:42:28.026594 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.017880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:48327 (local address 127.19.74.254:38953)
0806 22:42:28.017928 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.017931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.018109 (+ 178us) server_negotiation.cc:407] Connection header received
0806 22:42:28.018655 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.018656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.018668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.018732 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:28.019696 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.020198 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.021185 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.021388 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.021477 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.023584 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:28.023586 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:28.023586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:28.023593 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:28.025124 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:28.025603 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:28.025604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:28.025604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:28.025670 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:28.025899 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:28.025900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:28.025900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:28.026050 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:28.026134 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.026355 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.026460 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:28.026590 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.017843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38953 (local address 127.19.74.193:48327)
0806 22:42:28.017976 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.018025 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:28.018528 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:28.018804 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:28.018808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:28.019070 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:28.019593 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.019599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.020301 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.020302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.021093 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.021096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.021489 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.021490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.021625 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.022092 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:28.022102 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:28.023489 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:28.025197 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:28.025205 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:28.025208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:28.025508 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:28.025736 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:28.025736 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:28.025737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:28.025838 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:28.026118 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:28.026122 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:28.026232 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 22:42:28.026312 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:42:28.026427 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260806 22:42:28.027189 24183 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38953
I20260806 22:42:28.027300 24183 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:28.027468 24183 heartbeater.cc:511] Master 127.19.74.254:38953 requested a full tablet report, sending...
I20260806 22:42:28.028043 23985 ts_manager.cc:194] Registered new tserver with Master: e97cab8852fb4412940f45c683bca2cb (127.19.74.193:44599)
I20260806 22:42:28.028613 23985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:48327
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:28.051687 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38953
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:28.135569 24188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:28.135790 24188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:28.135830 24188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:28.137019 24188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:28.137094 24188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:28.137140 24188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:28.138693 24188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:28.139004 24188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:28.139263 24188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:28.144217 24196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:28.144234 24193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:28.144212 24194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:28.144490 24188 server_base.cc:1034] running on GCE node
I20260806 22:42:28.144695 24188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:28.144929 24188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:28.146080 24188 hybrid_clock.cc:648] HybridClock initialized: now 1786056148146067 us; error 28 us; skew 500 ppm
Aug 06 22:42:28 dist-test-slave-pgkr krb5kdc[23939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056148, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:28.148579 24188 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:28.148860 24188 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:28.148901 24188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:28.148986 24188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:28.149960 24188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "ba1c4c9492554df29d744fcc78a7ecaa"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "048b3e951741ada47c54b0a8ffba35c4"
server_key_iv: "9a23b7d1c071c7a21e43bcd5254b9af2"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.150270 24188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "ba1c4c9492554df29d744fcc78a7ecaa"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "048b3e951741ada47c54b0a8ffba35c4"
server_key_iv: "9a23b7d1c071c7a21e43bcd5254b9af2"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.151620 24188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:28.152415 24203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.152623 24188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:28.152736 24188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "ba1c4c9492554df29d744fcc78a7ecaa"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "048b3e951741ada47c54b0a8ffba35c4"
server_key_iv: "9a23b7d1c071c7a21e43bcd5254b9af2"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.152809 24188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:28.175352 24188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:28.177352 24188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:28.177520 24188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:28.177771 24188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:28.178100 24188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:28.178154 24188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.178195 24188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:28.178237 24188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.183930 24188 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:45975
I20260806 22:42:28.183966 24320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:45975 every 8 connection(s)
I20260806 22:42:28.184266 24188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:28.187044 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 24188
I20260806 22:42:28.187170 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:28.187398 19755 external_mini_cluster.cc:1428] Setting key 2ea114bf3d6b878e567e9a82d5901fee
Aug 06 22:42:28 dist-test-slave-pgkr krb5kdc[23939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056148, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:28.194558 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.184991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:59719 (local address 127.19.74.254:38953)
0806 22:42:28.185077 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.185080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.185248 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:42:28.185716 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.185717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.185727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.185779 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:28.186725 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.187295 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.188654 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.188854 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.188911 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.191555 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:28.191556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:28.191557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:28.191564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:28.193054 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:28.193559 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:28.193560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:28.193560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:28.193612 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:28.193903 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:28.193904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:28.193904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:28.194003 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:28.194053 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.194414 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.194483 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:42:28.194881 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.184951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38953 (local address 127.19.74.194:59719)
0806 22:42:28.185122 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.185158 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:28.185632 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:28.185833 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:28.185836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:28.186087 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:28.186577 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.186582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.187396 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.187397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.188557 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.188560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.188939 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.188940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.189094 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.189578 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:28.189588 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:28.191455 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:28.193143 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:28.193152 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:28.193155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:28.193478 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:28.193680 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:28.193681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:28.193682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:28.193831 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:28.194071 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:28.194079 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:28.194257 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 22:42:28.194388 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:42:28.194588 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260806 22:42:28.195446 24321 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38953
I20260806 22:42:28.195547 24321 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:28.195724 24321 heartbeater.cc:511] Master 127.19.74.254:38953 requested a full tablet report, sending...
I20260806 22:42:28.196086 23985 ts_manager.cc:194] Registered new tserver with Master: ba1c4c9492554df29d744fcc78a7ecaa (127.19.74.194:45975)
I20260806 22:42:28.196480 23985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:59719
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:28.213340 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38953
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:28.294731 24326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:28.294901 24326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:28.294924 24326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:28.296063 24326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:28.296098 24326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:28.296124 24326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:28.297573 24326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:28.297746 24326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:28.297905 24326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:28.302801 24331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:28.302795 24332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:28.302795 24334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:28.302882 24326 server_base.cc:1034] running on GCE node
I20260806 22:42:28.303192 24326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:28.303391 24326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:28.304580 24326 hybrid_clock.cc:648] HybridClock initialized: now 1786056148304537 us; error 54 us; skew 500 ppm
Aug 06 22:42:28 dist-test-slave-pgkr krb5kdc[23939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056148, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:28.307021 24326 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:28.307387 24326 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:28.307435 24326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:28.307526 24326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:28.308506 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "d7165bd81c1c4cdda1098aceccfbedc3"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "61e75b0ad7cc391018085a55b1eceb54"
server_key_iv: "3885011e432c54ea1fdd843115252d07"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.308854 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "d7165bd81c1c4cdda1098aceccfbedc3"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "61e75b0ad7cc391018085a55b1eceb54"
server_key_iv: "3885011e432c54ea1fdd843115252d07"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.310195 24326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:28.311082 24340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.311259 24326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.311359 24326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "d7165bd81c1c4cdda1098aceccfbedc3"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "61e75b0ad7cc391018085a55b1eceb54"
server_key_iv: "3885011e432c54ea1fdd843115252d07"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.311443 24326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:28.345566 24326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:28.347543 24326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:28.347699 24326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:28.347926 24326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:28.348242 24326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:28.348296 24326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.348337 24326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:28.348364 24326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.354385 24326 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:41847
I20260806 22:42:28.354425 24458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:41847 every 8 connection(s)
I20260806 22:42:28.354751 24326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:28.358405 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 24326
I20260806 22:42:28.358485 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:28.358700 19755 external_mini_cluster.cc:1428] Setting key 4bcd7120fde6133a3222707f9bc6c17e
Aug 06 22:42:28 dist-test-slave-pgkr krb5kdc[23939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056148, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:28.364550 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.355476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:52423 (local address 127.19.74.254:38953)
0806 22:42:28.355541 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.355543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.355790 (+ 247us) server_negotiation.cc:407] Connection header received
0806 22:42:28.356330 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.356331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.356345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.356402 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:28.357443 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.357968 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.359325 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.359642 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.359748 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.361807 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:28.361808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:28.361809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:28.361817 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:28.363123 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:28.363519 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:28.363520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:28.363521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:28.363583 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:28.363859 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:28.363860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:28.363860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:28.364026 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:28.364105 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.364376 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.364461 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:42:28.364760 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.355465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38953 (local address 127.19.74.195:52423)
0806 22:42:28.355674 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.355709 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:28.356209 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:28.356469 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:28.356473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:28.356824 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:28.357307 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.357312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.358065 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.358066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.359227 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.359230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.359760 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.359761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.359918 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.360381 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:28.360388 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:28.361711 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:28.363222 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:28.363229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:28.363231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:28.363436 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:28.363646 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:28.363647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:28.363647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:28.363764 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:28.364095 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:28.364102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:28.364235 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 22:42:28.364339 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:42:28.364527 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260806 22:42:28.365231 24459 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38953
I20260806 22:42:28.365334 24459 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:28.365492 24459 heartbeater.cc:511] Master 127.19.74.254:38953 requested a full tablet report, sending...
I20260806 22:42:28.365861 23985 ts_manager.cc:194] Registered new tserver with Master: d7165bd81c1c4cdda1098aceccfbedc3 (127.19.74.195:41847)
I20260806 22:42:28.366230 23985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:52423
I20260806 22:42:28.370662 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:28.377627 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.371913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55068 (local address 127.19.74.254:38953)
0806 22:42:28.371980 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.371982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.372083 (+ 101us) server_negotiation.cc:407] Connection header received
0806 22:42:28.372135 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.372135 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.372144 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.372196 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:28.372745 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.373218 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.374062 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.374243 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.374310 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.375460 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:28.375461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:28.375461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:28.375467 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:28.376680 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:28.376982 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:28.376983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:28.376983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:28.377033 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:28.377169 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:28.377169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:28.377170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:28.377297 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:28.377373 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.377495 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.377570 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:42:28.379272 23985 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:28.380872 23985 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:28.387141 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.383275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:45975 (local address 127.0.0.1:48568)
0806 22:42:28.383626 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.383633 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:28.383697 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:28.383886 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:28.383887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:28.383889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:28.383922 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.383923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.384757 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.384758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.385885 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.385888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.386807 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.386808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.386888 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.386905 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:42:28.386972 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:42:28.387062 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260806 22:42:28.387249 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.383198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.19.74.193:44599)
0806 22:42:28.383581 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.383585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.383594 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:28.383639 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.383641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.383654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.383725 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:28.383953 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.385099 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.386272 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.386856 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.386926 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.386982 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.387088 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.387139 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":80,"threads_started":1}
I20260806 22:42:28.387588 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.383378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.19.74.195:41847)
0806 22:42:28.383533 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.383538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.383549 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:28.383590 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.383593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.383609 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.383691 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:28.384077 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.385150 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.386329 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.387047 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.387426 (+ 379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.387471 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.387493 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.387522 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260806 22:42:28.387601 24475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.383408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.19.74.194:45975)
0806 22:42:28.383590 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.383594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.383750 (+ 156us) server_negotiation.cc:407] Connection header received
0806 22:42:28.383791 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.383793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.383809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.383889 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:28.383967 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.384654 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.385961 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.386720 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.386795 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.386850 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.387493 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.387532 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
I20260806 22:42:28.387280 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.383067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:44599 (local address 127.0.0.1:50416)
0806 22:42:28.383399 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.383408 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:28.383462 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:28.383818 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:28.383819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:28.383821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:28.383876 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.383878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.385205 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.385206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.386192 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.386194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.386930 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.386931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.386996 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.387001 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:28.387063 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:42:28.387135 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":39,"thread_start_us":81,"threads_started":1}
I20260806 22:42:28.387364 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.383067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:41847 (local address 127.0.0.1:39376)
0806 22:42:28.383227 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.383269 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:28.383336 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:28.383711 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:28.383714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:28.383716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:28.383780 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.383782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.385266 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.385268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.386237 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.386240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.387147 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.387148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.387239 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.387242 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:28.387272 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 22:42:28.387309 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":57,"threads_started":1}
I20260806 22:42:28.388470 24113 tablet_service.cc:1467] Processing CreateTablet for tablet cd346afb8c04403da51db1eb928818a6 (DEFAULT_TABLE table=test-table [id=31d6f913e8b94fe0a5d180cc05870a34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:28.388477 24389 tablet_service.cc:1467] Processing CreateTablet for tablet cd346afb8c04403da51db1eb928818a6 (DEFAULT_TABLE table=test-table [id=31d6f913e8b94fe0a5d180cc05870a34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:28.388481 24252 tablet_service.cc:1467] Processing CreateTablet for tablet cd346afb8c04403da51db1eb928818a6 (DEFAULT_TABLE table=test-table [id=31d6f913e8b94fe0a5d180cc05870a34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:28.388818 24113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd346afb8c04403da51db1eb928818a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:28.388818 24389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd346afb8c04403da51db1eb928818a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:28.388818 24252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd346afb8c04403da51db1eb928818a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:28.391443 24476 tablet_bootstrap.cc:492] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb: Bootstrap starting.
I20260806 22:42:28.392307 24476 tablet_bootstrap.cc:654] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:28.392246 24477 tablet_bootstrap.cc:492] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Bootstrap starting.
I20260806 22:42:28.392843 24477 tablet_bootstrap.cc:654] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:28.392836 24476 log.cc:824] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:28.393339 24477 log.cc:824] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:28.393666 24476 tablet_bootstrap.cc:492] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb: No bootstrap required, opened a new log
I20260806 22:42:28.393661 24478 tablet_bootstrap.cc:492] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Bootstrap starting.
I20260806 22:42:28.393756 24476 ts_tablet_manager.cc:1397] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:28.394205 24478 tablet_bootstrap.cc:654] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:28.394785 24478 log.cc:824] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:28.395093 24477 tablet_bootstrap.cc:492] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: No bootstrap required, opened a new log
I20260806 22:42:28.395195 24477 ts_tablet_manager.cc:1397] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:42:28.395468 24476 raft_consensus.cc:348] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.395614 24476 raft_consensus.cc:374] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:28.395673 24476 raft_consensus.cc:729] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97cab8852fb4412940f45c683bca2cb, State: Initialized, Role: FOLLOWER
I20260806 22:42:28.395787 24476 consensus_queue.cc:260] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.396090 24476 ts_tablet_manager.cc:1428] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:42:28.396134 24183 heartbeater.cc:503] Master 127.19.74.254:38953 was elected leader, sending a full tablet report...
I20260806 22:42:28.396831 24478 tablet_bootstrap.cc:492] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: No bootstrap required, opened a new log
I20260806 22:42:28.396903 24478 ts_tablet_manager.cc:1397] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 22:42:28.397085 24477 raft_consensus.cc:348] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.397238 24477 raft_consensus.cc:374] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:28.397276 24477 raft_consensus.cc:729] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba1c4c9492554df29d744fcc78a7ecaa, State: Initialized, Role: FOLLOWER
I20260806 22:42:28.397382 24477 consensus_queue.cc:260] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.397631 24477 ts_tablet_manager.cc:1428] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:28.397660 24321 heartbeater.cc:503] Master 127.19.74.254:38953 was elected leader, sending a full tablet report...
I20260806 22:42:28.398797 24478 raft_consensus.cc:348] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.398900 24478 raft_consensus.cc:374] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:28.398919 24478 raft_consensus.cc:729] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7165bd81c1c4cdda1098aceccfbedc3, State: Initialized, Role: FOLLOWER
I20260806 22:42:28.399052 24478 consensus_queue.cc:260] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.399278 24478 ts_tablet_manager.cc:1428] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:42:28.399349 24459 heartbeater.cc:503] Master 127.19.74.254:38953 was elected leader, sending a full tablet report...
W20260806 22:42:28.434804 24322 tablet.cc:2369] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:28.517899 24184 tablet.cc:2369] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:28.605286 24460 tablet.cc:2369] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:28.673327 24483 raft_consensus.cc:482] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:28.673503 24483 raft_consensus.cc:504] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.673990 24483 leader_election.cc:288] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7165bd81c1c4cdda1098aceccfbedc3 (127.19.74.195:41847), e97cab8852fb4412940f45c683bca2cb (127.19.74.193:44599)
I20260806 22:42:28.677898 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.674504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:40863 (local address 127.19.74.193:44599)
0806 22:42:28.674559 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.674561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.674570 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:28.674625 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.674627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.674637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.674688 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:28.674920 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.675564 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.677026 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.677518 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.677563 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.677571 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.677787 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.677825 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:42:28.678280 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.674172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:44599 (local address 127.19.74.194:40863)
0806 22:42:28.674368 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.674380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:28.674447 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:28.674745 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:28.674746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:28.674749 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:28.674845 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.674846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.675719 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.675720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.676921 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.676924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.677643 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.677644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.677712 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.677715 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:28.678071 (+ 356us) client_negotiation.cc:240] Negotiation successful
0806 22:42:28.678132 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":96,"thread_start_us":71,"threads_started":1}
I20260806 22:42:28.678781 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.674406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:40807 (local address 127.19.74.195:41847)
0806 22:42:28.674472 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.674474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.674485 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:28.674526 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.674527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.674537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.674661 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:28.674847 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.675549 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.677129 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.677783 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.677856 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.677865 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.678646 (+ 781us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.678696 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:42:28.678817 24131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd346afb8c04403da51db1eb928818a6" candidate_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97cab8852fb4412940f45c683bca2cb" is_pre_election: true
I20260806 22:42:28.678977 24131 raft_consensus.cc:2454] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba1c4c9492554df29d744fcc78a7ecaa in term 0.
I20260806 22:42:28.678287 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.674173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:41847 (local address 127.19.74.194:40807)
0806 22:42:28.674239 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.674251 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:28.674296 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:28.674658 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:28.674659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:28.674661 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:28.674740 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.674741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.675697 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.675698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.677012 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:28.677015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.677901 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.677903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:28.678011 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.678015 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:28.678072 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:42:28.678144 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":46}
I20260806 22:42:28.679214 24205 leader_election.cc:302] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba1c4c9492554df29d744fcc78a7ecaa, e97cab8852fb4412940f45c683bca2cb; no voters:
I20260806 22:42:28.679351 24483 raft_consensus.cc:2781] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:28.679423 24483 raft_consensus.cc:482] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:28.679454 24483 raft_consensus.cc:3037] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:28.679404 24409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd346afb8c04403da51db1eb928818a6" candidate_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" is_pre_election: true
I20260806 22:42:28.679538 24409 raft_consensus.cc:2454] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba1c4c9492554df29d744fcc78a7ecaa in term 0.
I20260806 22:42:28.680464 24483 raft_consensus.cc:504] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.680617 24483 leader_election.cc:288] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [CANDIDATE]: Term 1 election: Requested vote from peers d7165bd81c1c4cdda1098aceccfbedc3 (127.19.74.195:41847), e97cab8852fb4412940f45c683bca2cb (127.19.74.193:44599)
I20260806 22:42:28.680799 24409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd346afb8c04403da51db1eb928818a6" candidate_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7165bd81c1c4cdda1098aceccfbedc3"
I20260806 22:42:28.680893 24409 raft_consensus.cc:3037] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:28.680903 24131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd346afb8c04403da51db1eb928818a6" candidate_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97cab8852fb4412940f45c683bca2cb"
I20260806 22:42:28.680977 24131 raft_consensus.cc:3037] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:28.681628 24409 raft_consensus.cc:2454] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba1c4c9492554df29d744fcc78a7ecaa in term 1.
I20260806 22:42:28.681849 24131 raft_consensus.cc:2454] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba1c4c9492554df29d744fcc78a7ecaa in term 1.
I20260806 22:42:28.681840 24204 leader_election.cc:302] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba1c4c9492554df29d744fcc78a7ecaa, d7165bd81c1c4cdda1098aceccfbedc3; no voters:
I20260806 22:42:28.682076 24483 raft_consensus.cc:2781] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:28.682269 24483 raft_consensus.cc:686] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 1 LEADER]: Becoming Leader. State: Replica: ba1c4c9492554df29d744fcc78a7ecaa, State: Running, Role: LEADER
I20260806 22:42:28.682374 24483 consensus_queue.cc:237] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } }
I20260806 22:42:28.683310 23984 catalog_manager.cc:5495] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to ba1c4c9492554df29d744fcc78a7ecaa (127.19.74.194). New cstate: current_term: 1 leader_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba1c4c9492554df29d744fcc78a7ecaa" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 45975 } health_report { overall_health: HEALTHY } } }
I20260806 22:42:28.743665 24475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.740981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.19.74.194:45975)
0806 22:42:28.741045 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.741048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.741060 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:42:28.741110 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.741111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.741140 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.741224 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:28.741387 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.741873 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.742802 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:28.743019 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:28.743139 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:28.743279 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:28.743386 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:28.743473 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.743533 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.743555 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:42:28.745853 24409 raft_consensus.cc:1261] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 1 FOLLOWER]: Refusing update from remote peer ba1c4c9492554df29d744fcc78a7ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:28.745882 24131 raft_consensus.cc:1261] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 1 FOLLOWER]: Refusing update from remote peer ba1c4c9492554df29d744fcc78a7ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:28.746266 24486 consensus_queue.cc:1035] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7165bd81c1c4cdda1098aceccfbedc3" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:28.746425 24483 consensus_queue.cc:1035] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "e97cab8852fb4412940f45c683bca2cb" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:28.749725 24490 mvcc.cc:205] Tried to move back new op lower bound from 7315685985259847680 to 7315685985003651072. Current Snapshot: MvccSnapshot[applied={T|T < 7315685985259847680}]
I20260806 22:42:28.750403 24491 mvcc.cc:205] Tried to move back new op lower bound from 7315685985259847680 to 7315685985003651072. Current Snapshot: MvccSnapshot[applied={T|T < 7315685985259847680 or (T in {7315685985259847680})}]
I20260806 22:42:28.751431 23985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:28.751523 23985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:28.752440 23985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0103fa8aa81e453180fa750fd104f099: Sending DeleteTablet for 3 replicas of tablet cd346afb8c04403da51db1eb928818a6
I20260806 22:42:28.752873 24389 tablet_service.cc:1514] Processing DeleteTablet for tablet cd346afb8c04403da51db1eb928818a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:28 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:39376
I20260806 22:42:28.752908 24113 tablet_service.cc:1514] Processing DeleteTablet for tablet cd346afb8c04403da51db1eb928818a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:28 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:50416
I20260806 22:42:28.753113 24498 tablet_replica.cc:330] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: stopping tablet replica
I20260806 22:42:28.753150 24499 tablet_replica.cc:330] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb: stopping tablet replica
I20260806 22:42:28.753144 24252 tablet_service.cc:1514] Processing DeleteTablet for tablet cd346afb8c04403da51db1eb928818a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:28 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:48568
I20260806 22:42:28.753262 24499 raft_consensus.cc:2229] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:28.753293 24498 raft_consensus.cc:2229] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:28.753386 24499 raft_consensus.cc:2258] T cd346afb8c04403da51db1eb928818a6 P e97cab8852fb4412940f45c683bca2cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:28.753520 24498 raft_consensus.cc:2258] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:28.753556 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 24048
I20260806 22:42:28.755112 24498 ts_tablet_manager.cc:1905] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:28.757114 24498 ts_tablet_manager.cc:1918] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:28.757256 24498 log.cc:1196] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/cd346afb8c04403da51db1eb928818a6
I20260806 22:42:28.757638 24498 ts_tablet_manager.cc:1939] T cd346afb8c04403da51db1eb928818a6 P d7165bd81c1c4cdda1098aceccfbedc3: Deleting consensus metadata
I20260806 22:42:28.758103 23968 catalog_manager.cc:4841] TS d7165bd81c1c4cdda1098aceccfbedc3 (127.19.74.195:41847): tablet cd346afb8c04403da51db1eb928818a6 (table test-table [id=31d6f913e8b94fe0a5d180cc05870a34]) successfully deleted
I20260806 22:42:28.758879 24500 tablet_replica.cc:330] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: stopping tablet replica
I20260806 22:42:28.759038 24500 raft_consensus.cc:2229] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:28.759217 24500 raft_consensus.cc:2258] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:28.759721 24500 ts_tablet_manager.cc:1905] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:28.761513 24500 ts_tablet_manager.cc:1918] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:28.761654 24500 log.cc:1196] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/cd346afb8c04403da51db1eb928818a6
I20260806 22:42:28.761988 24500 ts_tablet_manager.cc:1939] T cd346afb8c04403da51db1eb928818a6 P ba1c4c9492554df29d744fcc78a7ecaa: Deleting consensus metadata
I20260806 22:42:28.762455 23969 catalog_manager.cc:4841] TS ba1c4c9492554df29d744fcc78a7ecaa (127.19.74.194:45975): tablet cd346afb8c04403da51db1eb928818a6 (table test-table [id=31d6f913e8b94fe0a5d180cc05870a34]) successfully deleted
W20260806 22:42:28.763724 23969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:44599 (error 108)
I20260806 22:42:28.764269 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 24188
I20260806 22:42:28.767015 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.766863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:44599 (local address 127.0.0.1:50420)
0806 22:42:28.766922 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:28.766959 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:44599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 22:42:28.767186 23969 catalog_manager.cc:4584] TS e97cab8852fb4412940f45c683bca2cb (127.19.74.193:44599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cd346afb8c04403da51db1eb928818a6: Network error: Client connection negotiation failed: client connection to 127.19.74.193:44599: connect: Connection refused (error 111)
I20260806 22:42:28.773909 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 24326
I20260806 22:42:28.780383 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 23952
2026-08-06T22:42:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2228 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T22:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:28Z Disabled control of system clock
I20260806 22:42:28.809274 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:44959
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:42227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:44959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:42:28.890362 24505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:28.890556 24505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:28.890576 24505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:28.891567 24505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:28.891599 24505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:28.891611 24505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:28.891623 24505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:28.891633 24505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:28.892845 24505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:44959
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:44959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:28.893059 24505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:28.893384 24505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:28.898056 24513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:28.898051 24510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:28.898051 24511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:28.898144 24505 server_base.cc:1034] running on GCE node
I20260806 22:42:28.898483 24505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:28.898838 24505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:28.899987 24505 hybrid_clock.cc:648] HybridClock initialized: now 1786056148899968 us; error 34 us; skew 500 ppm
I20260806 22:42:28.901324 24505 webserver.cc:460] Webserver started at http://127.19.74.254:33777/ using document root <none> and password file <none>
I20260806 22:42:28.901603 24505 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:28.901654 24505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:28.901804 24505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:28.902969 24505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "aaeb6696c1274379aaa279c62f1c84a7"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "007b2e6f77c8c133aca5f725b667cdc2"
server_key_iv: "79f6f74bb66ec194198767fc19669563"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.903368 24505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "aaeb6696c1274379aaa279c62f1c84a7"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "007b2e6f77c8c133aca5f725b667cdc2"
server_key_iv: "79f6f74bb66ec194198767fc19669563"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.904608 24505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:28.905387 24519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:28.905566 24505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:28.905674 24505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "aaeb6696c1274379aaa279c62f1c84a7"
format_stamp: "Formatted at 2026-08-06 22:42:28 on dist-test-slave-pgkr"
server_key: "007b2e6f77c8c133aca5f725b667cdc2"
server_key_iv: "79f6f74bb66ec194198767fc19669563"
server_key_version: "encryptionkey@0"
I20260806 22:42:28.905757 24505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:28.909111 24505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:28.909361 24505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:28.909502 24505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:28.913480 24505 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:44959
I20260806 22:42:28.913532 24573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:44959 every 8 connection(s)
I20260806 22:42:28.913849 24505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:28.914448 24574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:28.917208 24574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7: Bootstrap starting.
I20260806 22:42:28.917790 24574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:28.918079 24574 log.cc:824] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:28.918795 24574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7: No bootstrap required, opened a new log
I20260806 22:42:28.920349 24574 raft_consensus.cc:348] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaeb6696c1274379aaa279c62f1c84a7" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44959 } }
I20260806 22:42:28.920504 24574 raft_consensus.cc:374] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:28.920557 24574 raft_consensus.cc:729] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaeb6696c1274379aaa279c62f1c84a7, State: Initialized, Role: FOLLOWER
I20260806 22:42:28.920703 24574 consensus_queue.cc:260] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaeb6696c1274379aaa279c62f1c84a7" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44959 } }
I20260806 22:42:28.920789 24574 raft_consensus.cc:388] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:28.920831 24574 raft_consensus.cc:482] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:28.920881 24574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:28.921401 24574 raft_consensus.cc:504] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaeb6696c1274379aaa279c62f1c84a7" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44959 } }
I20260806 22:42:28.921520 24574 leader_election.cc:302] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aaeb6696c1274379aaa279c62f1c84a7; no voters:
I20260806 22:42:28.921700 24574 leader_election.cc:288] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:28.921744 24577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:28.921921 24577 raft_consensus.cc:686] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [term 1 LEADER]: Becoming Leader. State: Replica: aaeb6696c1274379aaa279c62f1c84a7, State: Running, Role: LEADER
I20260806 22:42:28.921954 24574 sys_catalog.cc:564] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:28.922065 24577 consensus_queue.cc:237] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaeb6696c1274379aaa279c62f1c84a7" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44959 } }
I20260806 22:42:28.922505 24579 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aaeb6696c1274379aaa279c62f1c84a7. Latest consensus state: current_term: 1 leader_uuid: "aaeb6696c1274379aaa279c62f1c84a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaeb6696c1274379aaa279c62f1c84a7" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44959 } } }
I20260806 22:42:28.922590 24579 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:28.922732 24578 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aaeb6696c1274379aaa279c62f1c84a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaeb6696c1274379aaa279c62f1c84a7" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 44959 } } }
I20260806 22:42:28.922847 24578 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:28.923022 24585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:28.923674 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 24505
I20260806 22:42:28.923692 24585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:28.923775 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:28.923956 19755 external_mini_cluster.cc:1428] Setting key 2a5104455de2eb19868fdd0f9c4de7e8
I20260806 22:42:28.925637 24585 catalog_manager.cc:1347] Generated new cluster ID: b0f368c835d147cd909c3dd752ea3710
I20260806 22:42:28.925730 24585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:28.927419 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:28.924556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48318 (local address 127.19.74.254:44959)
0806 22:42:28.924753 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:28.924763 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:28.924795 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:42:28.924920 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:28.924925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:28.924991 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:28.925142 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:28.926840 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:28.926843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:28.926844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:28.926847 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:28.926888 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:28.926903 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:28.927114 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:28.927146 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:42:28.927224 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260806 22:42:28.932432 24585 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:28.932578 24585 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:28.936707 24585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7: Generated new TSK 0
I20260806 22:42:28.936949 24585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:42:28.942262 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44959
--builtin_ntp_servers=127.19.74.212:42227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:42:29.025317 24598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:29.025521 24598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:29.025544 24598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:29.026819 24598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:29.026857 24598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:29.026885 24598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:29.028280 24598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:29.028520 24598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:29.028820 24598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:29.034554 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:29.034749 24606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:29.034564 24603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.034881 24598 server_base.cc:1034] running on GCE node
I20260806 22:42:29.035171 24598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:29.035418 24598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:29.036587 24598 hybrid_clock.cc:648] HybridClock initialized: now 1786056149036561 us; error 45 us; skew 500 ppm
I20260806 22:42:29.037969 24598 webserver.cc:460] Webserver started at http://127.19.74.193:33763/ using document root <none> and password file <none>
I20260806 22:42:29.038229 24598 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:29.038300 24598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:29.038443 24598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:29.039667 24598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "a6a1b51e892e4aa59c7a65c52fe5220e"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "4c12495fed6269478e82fc5c3a0769e9"
server_key_iv: "368140c5338622018d6518deb48f8b17"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.040051 24598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "a6a1b51e892e4aa59c7a65c52fe5220e"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "4c12495fed6269478e82fc5c3a0769e9"
server_key_iv: "368140c5338622018d6518deb48f8b17"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.041499 24598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:29.042302 24612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.042474 24598 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:29.042577 24598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "a6a1b51e892e4aa59c7a65c52fe5220e"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "4c12495fed6269478e82fc5c3a0769e9"
server_key_iv: "368140c5338622018d6518deb48f8b17"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.042690 24598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:29.045958 24598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:29.046214 24598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:29.046355 24598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:29.046559 24598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:29.046914 24598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:29.046970 24598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.047011 24598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:29.047036 24598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.052963 24598 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:34799
I20260806 22:42:29.053035 24727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:34799 every 8 connection(s)
I20260806 22:42:29.053341 24598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:29.056246 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.054066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:56811 (local address 127.19.74.254:44959)
0806 22:42:29.054116 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.054118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.054327 (+ 209us) server_negotiation.cc:407] Connection header received
0806 22:42:29.054863 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.054863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.054869 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.054929 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.056023 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.056024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.056024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.056029 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.056043 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.056048 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.056096 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.056175 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.056188 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:29.056414 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.054025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:44959 (local address 127.19.74.193:56811)
0806 22:42:29.054203 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.054242 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.054773 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.055003 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.055006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.055014 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:29.055880 (+ 866us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:42:29.055880 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:29.055906 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:42:29.055932 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:29.056116 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:29.056121 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:29.056123 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.056177 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.056226 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 22:42:29.056927 24728 heartbeater.cc:347] Connected to a master server at 127.19.74.254:44959
I20260806 22:42:29.057034 24728 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:29.057190 24728 heartbeater.cc:511] Master 127.19.74.254:44959 requested a full tablet report, sending...
I20260806 22:42:29.057791 24537 ts_manager.cc:194] Registered new tserver with Master: a6a1b51e892e4aa59c7a65c52fe5220e (127.19.74.193:34799)
I20260806 22:42:29.057806 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 24598
I20260806 22:42:29.057906 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:29.058084 19755 external_mini_cluster.cc:1428] Setting key 66386375c748436da4a8d676102d43c3
I20260806 22:42:29.059080 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44959
--builtin_ntp_servers=127.19.74.212:42227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:42:29.142189 24731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:29.142450 24731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:29.142490 24731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:29.143778 24731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:29.143844 24731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:29.143887 24731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:29.145365 24731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:29.145622 24731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:29.145902 24731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:29.151063 24737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:29.151074 24736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.151363 24731 server_base.cc:1034] running on GCE node
W20260806 22:42:29.151074 24739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.151670 24731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:29.151902 24731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:29.153039 24731 hybrid_clock.cc:648] HybridClock initialized: now 1786056149153024 us; error 30 us; skew 500 ppm
I20260806 22:42:29.154229 24731 webserver.cc:460] Webserver started at http://127.19.74.194:33481/ using document root <none> and password file <none>
I20260806 22:42:29.154407 24731 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:29.154444 24731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:29.154531 24731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:29.155613 24731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "5015689df11040679fe19d96c0a58471"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "c69eb4f323e81faefd9bf583531580b3"
server_key_iv: "1f80e508023e6dd43684f4402f47c871"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.155972 24731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "5015689df11040679fe19d96c0a58471"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "c69eb4f323e81faefd9bf583531580b3"
server_key_iv: "1f80e508023e6dd43684f4402f47c871"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.157378 24731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:42:29.158053 24745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.158216 24731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:29.158308 24731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "5015689df11040679fe19d96c0a58471"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "c69eb4f323e81faefd9bf583531580b3"
server_key_iv: "1f80e508023e6dd43684f4402f47c871"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.158362 24731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:29.161600 24731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:29.161794 24731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:29.161880 24731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:29.162022 24731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:29.162263 24731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:29.162292 24731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.162310 24731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:29.162322 24731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.167779 24731 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:43713
I20260806 22:42:29.167868 24859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:43713 every 8 connection(s)
I20260806 22:42:29.168146 24731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:29.170890 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.168816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:60205 (local address 127.19.74.254:44959)
0806 22:42:29.168858 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.168860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.169089 (+ 229us) server_negotiation.cc:407] Connection header received
0806 22:42:29.169478 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.169479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.169483 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.169539 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.170657 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.170658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.170659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.170664 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.170687 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.170692 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.170741 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.170826 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.170838 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:42:29.171080 24862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.168777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:44959 (local address 127.19.74.194:60205)
0806 22:42:29.168968 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.169004 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.169402 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.169623 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.169626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.169634 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:29.170483 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:42:29.170484 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:29.170509 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:42:29.170534 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:29.170754 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:29.170759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:29.170761 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.170822 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.170869 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260806 22:42:29.171571 24860 heartbeater.cc:347] Connected to a master server at 127.19.74.254:44959
I20260806 22:42:29.171684 24860 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:29.171855 24860 heartbeater.cc:511] Master 127.19.74.254:44959 requested a full tablet report, sending...
I20260806 22:42:29.172266 24537 ts_manager.cc:194] Registered new tserver with Master: 5015689df11040679fe19d96c0a58471 (127.19.74.194:43713)
I20260806 22:42:29.174201 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 24731
I20260806 22:42:29.174309 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:29.174480 19755 external_mini_cluster.cc:1428] Setting key ecb49ed909c23584d7b1dfa9793faa99
I20260806 22:42:29.175468 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44959
--builtin_ntp_servers=127.19.74.212:42227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:42:29.260296 24863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:29.260494 24863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:29.260514 24863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:29.261694 24863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:29.261735 24863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:29.261795 24863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:29.263221 24863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:44959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:29.263450 24863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:29.263746 24863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:29.268530 24868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:29.268551 24871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:29.268587 24869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.268792 24863 server_base.cc:1034] running on GCE node
I20260806 22:42:29.269039 24863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:29.269300 24863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:29.270481 24863 hybrid_clock.cc:648] HybridClock initialized: now 1786056149270451 us; error 42 us; skew 500 ppm
I20260806 22:42:29.271819 24863 webserver.cc:460] Webserver started at http://127.19.74.195:44851/ using document root <none> and password file <none>
I20260806 22:42:29.272082 24863 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:29.272159 24863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:29.272321 24863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:29.273383 24863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "f7ceafa6c08b4bcbaf564abf977511fc"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "b3652a40583990ba71dfb3be54c4d944"
server_key_iv: "6b7fe7fcddb4b28cf8be6529be1538c5"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.273785 24863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "f7ceafa6c08b4bcbaf564abf977511fc"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "b3652a40583990ba71dfb3be54c4d944"
server_key_iv: "6b7fe7fcddb4b28cf8be6529be1538c5"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.275255 24863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:29.275941 24877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.276115 24863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:29.276209 24863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "f7ceafa6c08b4bcbaf564abf977511fc"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "b3652a40583990ba71dfb3be54c4d944"
server_key_iv: "6b7fe7fcddb4b28cf8be6529be1538c5"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.276288 24863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:29.279608 24863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:29.279877 24863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:29.280011 24863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:29.280218 24863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:29.280561 24863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:29.280609 24863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.280649 24863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:29.280674 24863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.286031 24863 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:36531
I20260806 22:42:29.286067 24994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:36531 every 8 connection(s)
I20260806 22:42:29.286394 24863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:29.290761 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 24863
I20260806 22:42:29.290865 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:29.291033 19755 external_mini_cluster.cc:1428] Setting key 994f006a7213ba905bf599947eeef36e
I20260806 22:42:29.292707 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.287156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:58835 (local address 127.19.74.254:44959)
0806 22:42:29.287215 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.287217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.287414 (+ 197us) server_negotiation.cc:407] Connection header received
0806 22:42:29.287817 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.287818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.287823 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.287889 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.292504 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.292505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.292505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.292510 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.292525 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.292530 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.292578 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.292643 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.292656 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:42:29.292855 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.287108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:44959 (local address 127.19.74.195:58835)
0806 22:42:29.287292 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.287328 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.287734 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.287936 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.287939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.287946 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:29.292356 (+ 4410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:42:29.292357 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:29.292383 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:42:29.292409 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:29.292576 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:29.292578 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:29.292580 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.292632 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.292676 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260806 22:42:29.293330 24995 heartbeater.cc:347] Connected to a master server at 127.19.74.254:44959
I20260806 22:42:29.293422 24995 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:29.293529 24995 heartbeater.cc:511] Master 127.19.74.254:44959 requested a full tablet report, sending...
I20260806 22:42:29.293886 24537 ts_manager.cc:194] Registered new tserver with Master: f7ceafa6c08b4bcbaf564abf977511fc (127.19.74.195:36531)
I20260806 22:42:29.302639 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:29.304910 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.304012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.19.74.254:44959)
0806 22:42:29.304078 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.304080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.304144 (+ 64us) server_negotiation.cc:407] Connection header received
0806 22:42:29.304179 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.304180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.304184 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.304234 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.304707 (+ 473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.304708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.304708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.304712 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.304725 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.304728 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.304776 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.304810 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.304821 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:42:29.306322 24537 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:29.307897 24537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:29.311357 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.309849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:36531 (local address 127.0.0.1:37718)
0806 22:42:29.309963 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.310001 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.310056 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.310361 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.310367 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.310374 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:29.310895 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:42:29.310896 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:29.310911 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:42:29.310933 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:29.311205 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:29.311212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:29.311215 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.311261 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.311275 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260806 22:42:29.311510 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.310124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.19.74.194:43713)
0806 22:42:29.310320 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.310324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.310335 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:29.310366 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.310368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.310377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.310449 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.311144 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.311150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.311151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.311155 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.311187 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.311206 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.311260 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.311392 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.311409 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260806 22:42:29.311556 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.309858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:43713 (local address 127.0.0.1:50796)
0806 22:42:29.310283 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.310292 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.310323 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.310675 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.310676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.310680 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:29.311045 (+ 365us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:42:29.311046 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:29.311052 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:42:29.311060 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:29.311285 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:29.311285 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:29.311286 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.311327 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.311337 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":16,"thread_start_us":104,"threads_started":1}
I20260806 22:42:29.311780 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.310016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37718 (local address 127.19.74.195:36531)
0806 22:42:29.310184 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.310188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.310196 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:29.310214 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.310215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.310243 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.310571 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.311056 (+ 485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.311060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.311060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.311064 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.311096 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.311124 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.311665 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.311691 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.311707 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260806 22:42:29.311921 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.309857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:34799 (local address 127.0.0.1:53522)
0806 22:42:29.310143 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.310153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.310202 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.310690 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.310692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.310695 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:29.311215 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:42:29.311215 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:29.311222 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:42:29.311236 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:29.311788 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:29.311789 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:29.311790 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.311825 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.311838 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":125,"threads_started":1}
I20260806 22:42:29.312327 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.310322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53522 (local address 127.19.74.193:34799)
0806 22:42:29.310476 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.310480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.310490 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:29.310517 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.310523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.310532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.310862 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.311323 (+ 461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.311331 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.311331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.311336 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.311376 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.311400 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.312164 (+ 764us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.312189 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.312206 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 22:42:29.312348 24793 tablet_service.cc:1467] Processing CreateTablet for tablet b92ae84d6be54625aaedaf474d762bdc (DEFAULT_TABLE table=test-table [id=6df8b3394320405ea3f07668c648a834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:29.312405 24927 tablet_service.cc:1467] Processing CreateTablet for tablet b92ae84d6be54625aaedaf474d762bdc (DEFAULT_TABLE table=test-table [id=6df8b3394320405ea3f07668c648a834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:29.312754 24793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92ae84d6be54625aaedaf474d762bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:29.312785 24927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92ae84d6be54625aaedaf474d762bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:29.315271 25009 tablet_bootstrap.cc:492] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: Bootstrap starting.
I20260806 22:42:29.315410 25010 tablet_bootstrap.cc:492] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: Bootstrap starting.
I20260806 22:42:29.316095 25009 tablet_bootstrap.cc:654] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:29.316182 25010 tablet_bootstrap.cc:654] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:29.316121 24660 tablet_service.cc:1467] Processing CreateTablet for tablet b92ae84d6be54625aaedaf474d762bdc (DEFAULT_TABLE table=test-table [id=6df8b3394320405ea3f07668c648a834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:29.316422 25009 log.cc:824] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:29.316478 24660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92ae84d6be54625aaedaf474d762bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:29.316545 25010 log.cc:824] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:29.317193 25009 tablet_bootstrap.cc:492] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: No bootstrap required, opened a new log
I20260806 22:42:29.317226 25010 tablet_bootstrap.cc:492] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: No bootstrap required, opened a new log
I20260806 22:42:29.317245 25009 ts_tablet_manager.cc:1397] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:29.317268 25010 ts_tablet_manager.cc:1397] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:29.319094 25010 raft_consensus.cc:348] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.319233 25010 raft_consensus.cc:374] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:29.319307 25010 raft_consensus.cc:729] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5015689df11040679fe19d96c0a58471, State: Initialized, Role: FOLLOWER
I20260806 22:42:29.319453 25010 consensus_queue.cc:260] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.319726 25010 ts_tablet_manager.cc:1428] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:42:29.319646 25009 raft_consensus.cc:348] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.319795 25009 raft_consensus.cc:374] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:29.319847 25009 raft_consensus.cc:729] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ceafa6c08b4bcbaf564abf977511fc, State: Initialized, Role: FOLLOWER
I20260806 22:42:29.319944 25009 consensus_queue.cc:260] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.320127 25009 ts_tablet_manager.cc:1428] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:42:29.320176 24995 heartbeater.cc:503] Master 127.19.74.254:44959 was elected leader, sending a full tablet report...
I20260806 22:42:29.320254 24860 heartbeater.cc:503] Master 127.19.74.254:44959 was elected leader, sending a full tablet report...
I20260806 22:42:29.320577 25013 tablet_bootstrap.cc:492] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e: Bootstrap starting.
I20260806 22:42:29.321281 25013 tablet_bootstrap.cc:654] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:29.321568 25013 log.cc:824] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:29.322304 25013 tablet_bootstrap.cc:492] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e: No bootstrap required, opened a new log
I20260806 22:42:29.322386 25013 ts_tablet_manager.cc:1397] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:29.324210 25013 raft_consensus.cc:348] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.324347 25013 raft_consensus.cc:374] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:29.324391 25013 raft_consensus.cc:729] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6a1b51e892e4aa59c7a65c52fe5220e, State: Initialized, Role: FOLLOWER
I20260806 22:42:29.324503 25013 consensus_queue.cc:260] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.324726 25013 ts_tablet_manager.cc:1428] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:42:29.324808 24728 heartbeater.cc:503] Master 127.19.74.254:44959 was elected leader, sending a full tablet report...
I20260806 22:42:29.343586 25015 raft_consensus.cc:482] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:29.343703 25015 raft_consensus.cc:504] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.343973 25015 leader_election.cc:288] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5015689df11040679fe19d96c0a58471 (127.19.74.194:43713), a6a1b51e892e4aa59c7a65c52fe5220e (127.19.74.193:34799)
I20260806 22:42:29.345384 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.344108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:43713 (local address 127.19.74.195:58489)
0806 22:42:29.344153 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.344165 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.344203 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.344368 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.344369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.344371 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:29.344984 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:42:29.344985 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:29.344993 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:42:29.345006 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:29.345206 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:29.345207 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:29.345208 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.345280 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.345293 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:42:29.345434 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.344198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:58489 (local address 127.19.74.194:43713)
0806 22:42:29.344239 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.344242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.344248 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:42:29.344272 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.344273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.344278 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.344337 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.345086 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.345087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.345087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.345092 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.345115 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.345119 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.345194 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.345338 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.345353 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:29.345603 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.344224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:35909 (local address 127.19.74.193:34799)
0806 22:42:29.344275 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.344277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.344403 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:42:29.344427 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.344428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.344433 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.344488 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.345339 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.345340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.345341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.345345 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.345360 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.345365 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.345409 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.345530 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.345545 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:42:29.345849 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.344119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:34799 (local address 127.19.74.195:35909)
0806 22:42:29.344307 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.344316 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.344361 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.344643 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.344643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.344646 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:29.345185 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:42:29.345186 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:29.345193 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:42:29.345206 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:29.345444 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:29.345445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:29.345446 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.345729 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.345745 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":18,"thread_start_us":94,"threads_started":1}
I20260806 22:42:29.346060 24813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92ae84d6be54625aaedaf474d762bdc" candidate_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5015689df11040679fe19d96c0a58471" is_pre_election: true
I20260806 22:42:29.346212 24813 raft_consensus.cc:2454] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ceafa6c08b4bcbaf564abf977511fc in term 0.
I20260806 22:42:29.346207 24681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92ae84d6be54625aaedaf474d762bdc" candidate_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" is_pre_election: true
I20260806 22:42:29.346369 24681 raft_consensus.cc:2454] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ceafa6c08b4bcbaf564abf977511fc in term 0.
I20260806 22:42:29.346436 24880 leader_election.cc:302] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5015689df11040679fe19d96c0a58471, f7ceafa6c08b4bcbaf564abf977511fc; no voters:
I20260806 22:42:29.346547 25015 raft_consensus.cc:2781] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:29.346643 25015 raft_consensus.cc:482] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:29.346679 25015 raft_consensus.cc:3037] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:29.347342 25015 raft_consensus.cc:504] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.347478 25015 leader_election.cc:288] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [CANDIDATE]: Term 1 election: Requested vote from peers 5015689df11040679fe19d96c0a58471 (127.19.74.194:43713), a6a1b51e892e4aa59c7a65c52fe5220e (127.19.74.193:34799)
I20260806 22:42:29.347635 24813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92ae84d6be54625aaedaf474d762bdc" candidate_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5015689df11040679fe19d96c0a58471"
I20260806 22:42:29.347622 24681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92ae84d6be54625aaedaf474d762bdc" candidate_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e"
I20260806 22:42:29.347698 24813 raft_consensus.cc:3037] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:29.347698 24681 raft_consensus.cc:3037] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:29.348524 24813 raft_consensus.cc:2454] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ceafa6c08b4bcbaf564abf977511fc in term 1.
I20260806 22:42:29.348524 24681 raft_consensus.cc:2454] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ceafa6c08b4bcbaf564abf977511fc in term 1.
I20260806 22:42:29.348703 24879 leader_election.cc:302] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6a1b51e892e4aa59c7a65c52fe5220e, f7ceafa6c08b4bcbaf564abf977511fc; no voters:
I20260806 22:42:29.348803 25015 raft_consensus.cc:2781] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:29.348948 25015 raft_consensus.cc:686] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 1 LEADER]: Becoming Leader. State: Replica: f7ceafa6c08b4bcbaf564abf977511fc, State: Running, Role: LEADER
I20260806 22:42:29.349062 25015 consensus_queue.cc:237] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } }
I20260806 22:42:29.349833 24537 catalog_manager.cc:5495] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc reported cstate change: term changed from 0 to 1, leader changed from <none> to f7ceafa6c08b4bcbaf564abf977511fc (127.19.74.195). New cstate: current_term: 1 leader_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ceafa6c08b4bcbaf564abf977511fc" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:29.357184 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.356461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.19.74.195:36531)
0806 22:42:29.356502 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.356505 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.356512 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:29.356580 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.356580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.356590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.356638 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.357029 (+ 391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.357030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.357030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.357033 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.357044 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.357047 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.357092 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.357134 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.357144 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:29.358808 24681 raft_consensus.cc:1261] T b92ae84d6be54625aaedaf474d762bdc P a6a1b51e892e4aa59c7a65c52fe5220e [term 1 FOLLOWER]: Refusing update from remote peer f7ceafa6c08b4bcbaf564abf977511fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:29.358808 24813 raft_consensus.cc:1261] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 1 FOLLOWER]: Refusing update from remote peer f7ceafa6c08b4bcbaf564abf977511fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:29.359128 25015 consensus_queue.cc:1035] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a1b51e892e4aa59c7a65c52fe5220e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 34799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:29.359242 25019 consensus_queue.cc:1035] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5015689df11040679fe19d96c0a58471" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:29.361624 25026 mvcc.cc:205] Tried to move back new op lower bound from 7315685987770966016 to 7315685987734450176. Current Snapshot: MvccSnapshot[applied={T|T < 7315685987770966016}]
I20260806 22:42:29.362376 25027 mvcc.cc:205] Tried to move back new op lower bound from 7315685987770966016 to 7315685987734450176. Current Snapshot: MvccSnapshot[applied={T|T < 7315685987770966016}]
I20260806 22:42:29.363560 24537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:29.363649 24537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:29.364365 24537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aaeb6696c1274379aaa279c62f1c84a7: Sending DeleteTablet for 3 replicas of tablet b92ae84d6be54625aaedaf474d762bdc
I20260806 22:42:29.364693 24660 tablet_service.cc:1514] Processing DeleteTablet for tablet b92ae84d6be54625aaedaf474d762bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:29 UTC) from {username='slave'} at 127.0.0.1:53522
I20260806 22:42:29.364718 24793 tablet_service.cc:1514] Processing DeleteTablet for tablet b92ae84d6be54625aaedaf474d762bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:29 UTC) from {username='slave'} at 127.0.0.1:50796
I20260806 22:42:29.364909 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 24598
I20260806 22:42:29.364988 25035 tablet_replica.cc:330] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: stopping tablet replica
I20260806 22:42:29.364974 24927 tablet_service.cc:1514] Processing DeleteTablet for tablet b92ae84d6be54625aaedaf474d762bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:29 UTC) from {username='slave'} at 127.0.0.1:37718
I20260806 22:42:29.365072 25035 raft_consensus.cc:2229] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:29.365156 25035 raft_consensus.cc:2258] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:29.365535 25035 ts_tablet_manager.cc:1905] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:29.367751 25035 ts_tablet_manager.cc:1918] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:29.367882 25035 log.cc:1196] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/b92ae84d6be54625aaedaf474d762bdc
I20260806 22:42:29.368206 25035 ts_tablet_manager.cc:1939] T b92ae84d6be54625aaedaf474d762bdc P 5015689df11040679fe19d96c0a58471: Deleting consensus metadata
I20260806 22:42:29.368629 24521 catalog_manager.cc:4841] TS 5015689df11040679fe19d96c0a58471 (127.19.74.194:43713): tablet b92ae84d6be54625aaedaf474d762bdc (table test-table [id=6df8b3394320405ea3f07668c648a834]) successfully deleted
I20260806 22:42:29.370738 25036 tablet_replica.cc:330] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: stopping tablet replica
I20260806 22:42:29.370877 25036 raft_consensus.cc:2229] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:29.370975 25036 raft_consensus.cc:2258] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:29.371405 25036 ts_tablet_manager.cc:1905] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:29.373361 25036 ts_tablet_manager.cc:1918] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:29.373437 25036 log.cc:1196] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/b92ae84d6be54625aaedaf474d762bdc
I20260806 22:42:29.373723 25036 ts_tablet_manager.cc:1939] T b92ae84d6be54625aaedaf474d762bdc P f7ceafa6c08b4bcbaf564abf977511fc: Deleting consensus metadata
I20260806 22:42:29.374151 24524 catalog_manager.cc:4841] TS f7ceafa6c08b4bcbaf564abf977511fc (127.19.74.195:36531): tablet b92ae84d6be54625aaedaf474d762bdc (table test-table [id=6df8b3394320405ea3f07668c648a834]) successfully deleted
W20260806 22:42:29.374472 24520 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:34799 (error 108)
I20260806 22:42:29.374687 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 24731
I20260806 22:42:29.375564 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.375428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:34799 (local address 127.0.0.1:53530)
0806 22:42:29.375490 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.375534 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:34799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 22:42:29.375702 24520 catalog_manager.cc:4584] TS a6a1b51e892e4aa59c7a65c52fe5220e (127.19.74.193:34799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b92ae84d6be54625aaedaf474d762bdc: Network error: Client connection negotiation failed: client connection to 127.19.74.193:34799: connect: Connection refused (error 111)
I20260806 22:42:29.382973 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 24863
I20260806 22:42:29.388511 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 24505
2026-08-06T22:42:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (606 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 22:42:29.408674 19755 mini_oidc.cc:152] Starting JWKS server
I20260806 22:42:29.409750 19755 webserver.cc:460] Webserver started at https://127.0.0.1:42677/ using document root <none> and password file <none>
I20260806 22:42:29.409852 19755 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:42:29.410017 19755 webserver.cc:460] Webserver started at http://127.0.0.1:32789/ using document root <none> and password file <none>
2026-08-06T22:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:29Z Disabled control of system clock
I20260806 22:42:29.418443 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34687
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:46215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:34687
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42677/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/testchainca.cert with env {}
W20260806 22:42:29.496413 25044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:29.496661 25044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:29.496699 25044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:29.497870 25044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:29.497938 25044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:29.497957 25044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:29.497977 25044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:29.498006 25044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:42:29.498026 25044 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:42:29.498045 25044 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42677/jwks.json
I20260806 22:42:29.499516 25044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:34687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42677/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:29.499786 25044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:29.500003 25044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:29.504915 25050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:29.504899 25051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.504932 25044 server_base.cc:1034] running on GCE node
W20260806 22:42:29.504899 25053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.505380 25044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:29.505689 25044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:29.506846 25044 hybrid_clock.cc:648] HybridClock initialized: now 1786056149506835 us; error 27 us; skew 500 ppm
I20260806 22:42:29.508100 25044 webserver.cc:460] Webserver started at http://127.19.74.254:46275/ using document root <none> and password file <none>
I20260806 22:42:29.508291 25044 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:29.508330 25044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:29.508415 25044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:29.509367 25044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "31257df849ad4cfc8bb4a5603d7155da"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "0c5eeec36888abb1b23006b8128704bd"
server_key_iv: "ff20bbf14362152155a58e8abc8ef61e"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.509769 25044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "31257df849ad4cfc8bb4a5603d7155da"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "0c5eeec36888abb1b23006b8128704bd"
server_key_iv: "ff20bbf14362152155a58e8abc8ef61e"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.511147 25044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:29.511861 25059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.512040 25044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:29.512127 25044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "31257df849ad4cfc8bb4a5603d7155da"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "0c5eeec36888abb1b23006b8128704bd"
server_key_iv: "ff20bbf14362152155a58e8abc8ef61e"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.512179 25044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:29.528925 25044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:29.529152 25044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:29.529248 25044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:29.533160 25044 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:34687
I20260806 22:42:29.533221 25112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:34687 every 8 connection(s)
I20260806 22:42:29.533538 25044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:29.534539 25113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:29.536827 25113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da: Bootstrap starting.
I20260806 22:42:29.537369 25113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:29.537635 25113 log.cc:824] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:29.538349 25113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da: No bootstrap required, opened a new log
I20260806 22:42:29.539937 25113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31257df849ad4cfc8bb4a5603d7155da" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34687 } }
I20260806 22:42:29.540067 25113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:29.540091 25113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31257df849ad4cfc8bb4a5603d7155da, State: Initialized, Role: FOLLOWER
I20260806 22:42:29.540257 25113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31257df849ad4cfc8bb4a5603d7155da" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34687 } }
I20260806 22:42:29.540340 25113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:29.540383 25113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:29.540434 25113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:29.540961 25113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31257df849ad4cfc8bb4a5603d7155da" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34687 } }
I20260806 22:42:29.541072 25113 leader_election.cc:302] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31257df849ad4cfc8bb4a5603d7155da; no voters:
I20260806 22:42:29.541256 25113 leader_election.cc:288] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:29.541306 25116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:29.541497 25116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [term 1 LEADER]: Becoming Leader. State: Replica: 31257df849ad4cfc8bb4a5603d7155da, State: Running, Role: LEADER
I20260806 22:42:29.541535 25113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:29.541689 25116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31257df849ad4cfc8bb4a5603d7155da" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34687 } }
I20260806 22:42:29.542104 25118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31257df849ad4cfc8bb4a5603d7155da. Latest consensus state: current_term: 1 leader_uuid: "31257df849ad4cfc8bb4a5603d7155da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31257df849ad4cfc8bb4a5603d7155da" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34687 } } }
I20260806 22:42:29.542174 25117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31257df849ad4cfc8bb4a5603d7155da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31257df849ad4cfc8bb4a5603d7155da" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34687 } } }
I20260806 22:42:29.542196 25118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:29.542222 25117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:29.542251 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 25044
I20260806 22:42:29.542337 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:29.542388 25120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:29.542538 19755 external_mini_cluster.cc:1428] Setting key 2674c4e942a2819b981a2c9238ad2e97
I20260806 22:42:29.543169 25120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:29.544551 25120 catalog_manager.cc:1347] Generated new cluster ID: 9b8d0cd47a4a4b08a65799bd974d3d36
I20260806 22:42:29.544608 25120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:29.549031 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.543041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43908 (local address 127.19.74.254:34687)
0806 22:42:29.543223 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.543232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.543838 (+ 606us) server_negotiation.cc:407] Connection header received
0806 22:42:29.543997 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.544005 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.544071 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.544205 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.544532 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.545830 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.546938 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.547576 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.547702 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.548339 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.548343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.548344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.548546 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.548598 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.548612 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.548683 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.548749 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.548825 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":84,"threads_started":1}
I20260806 22:42:29.550024 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:42:29.552450 25120 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:29.552925 25120 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:29.558090 25120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31257df849ad4cfc8bb4a5603d7155da: Generated new TSK 0
I20260806 22:42:29.558259 25120 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:42:29.586658 25145 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 22:42:29.586759 25145 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260806 22:42:29.587271 25144 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:29.563980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:34687 (local address 127.0.0.1:43914)
0806 22:42:29.564217 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.564228 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.564297 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.564478 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.564479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.564481 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:42:29.564542 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.564545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.565135 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.565136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.566040 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.566042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.566359 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.566360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.566441 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.566463 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:42:29.586866 (+ 20403us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:42:29.586967 (+ 101us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.74.254:34687: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260806 22:42:29.587563 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.564076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43914 (local address 127.19.74.254:34687)
0806 22:42:29.564125 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.564127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.564294 (+ 167us) server_negotiation.cc:407] Connection header received
0806 22:42:29.564389 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.564390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.564399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.564453 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:42:29.564604 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.565037 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.566123 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.566289 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.566337 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.566568 (+ 231us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:42:29.586696 (+ 20128us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:42:29.587504 (+ 808us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43914: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:42:29.587678 25129 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43914: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.74.254:34687: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 22:42:29.587844 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 25044
2026-08-06T22:42:29Z chronyd exiting
I20260806 22:42:29.625571 19755 test_util.cc:175] -----------------------------------------------
I20260806 22:42:29.625687 19755 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056127658350-19755-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (218 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 22:42:29.626492 19755 mini_oidc.cc:152] Starting JWKS server
I20260806 22:42:29.627244 19755 webserver.cc:460] Webserver started at https://127.0.0.1:33647/ using document root <none> and password file <none>
I20260806 22:42:29.627313 19755 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:42:29.627477 19755 webserver.cc:460] Webserver started at http://127.0.0.1:44945/ using document root <none> and password file <none>
2026-08-06T22:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:29Z Disabled control of system clock
I20260806 22:42:29.636973 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37437
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:45113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:37437
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33647/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/testchainca.cert with env {}
W20260806 22:42:29.718116 25151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:29.718307 25151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:29.718328 25151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:29.719465 25151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:29.719506 25151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:29.719519 25151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:29.719568 25151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:29.719604 25151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:42:29.719631 25151 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:42:29.719657 25151 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33647/jwks.json
I20260806 22:42:29.721196 25151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:37437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33647/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:29.721458 25151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:29.721686 25151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:29.726373 25156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:29.726399 25157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.726469 25151 server_base.cc:1034] running on GCE node
W20260806 22:42:29.726399 25159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.726856 25151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:29.727176 25151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:29.728359 25151 hybrid_clock.cc:648] HybridClock initialized: now 1786056149728343 us; error 26 us; skew 500 ppm
I20260806 22:42:29.729660 25151 webserver.cc:460] Webserver started at http://127.19.74.254:36049/ using document root <none> and password file <none>
I20260806 22:42:29.729888 25151 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:29.729955 25151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:29.730098 25151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:29.731246 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "93e535e21e9c48e99068577160e80f50"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "6e967340794588e8c59199788f1bad4c"
server_key_iv: "b86ba86056033bf3e669391edcbd9ab3"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.731600 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "93e535e21e9c48e99068577160e80f50"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "6e967340794588e8c59199788f1bad4c"
server_key_iv: "b86ba86056033bf3e669391edcbd9ab3"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.732945 25151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:29.733633 25165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.733803 25151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:29.733904 25151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "93e535e21e9c48e99068577160e80f50"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "6e967340794588e8c59199788f1bad4c"
server_key_iv: "b86ba86056033bf3e669391edcbd9ab3"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.733973 25151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:29.750353 25151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:29.750705 25151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:29.750852 25151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:29.754765 25151 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:37437
I20260806 22:42:29.754796 25218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:37437 every 8 connection(s)
I20260806 22:42:29.755116 25151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:29.755795 25219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:29.758571 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50: Bootstrap starting.
I20260806 22:42:29.759245 25219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:29.759537 25219 log.cc:824] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:29.760190 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50: No bootstrap required, opened a new log
I20260806 22:42:29.761343 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 25151
I20260806 22:42:29.761426 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:29.761618 19755 external_mini_cluster.cc:1428] Setting key 44bc596a536fa2c2efbbb352a5318766
I20260806 22:42:29.761902 25219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e535e21e9c48e99068577160e80f50" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37437 } }
I20260806 22:42:29.762081 25219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:29.762197 25219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e535e21e9c48e99068577160e80f50, State: Initialized, Role: FOLLOWER
I20260806 22:42:29.762351 25219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e535e21e9c48e99068577160e80f50" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37437 } }
I20260806 22:42:29.762441 25219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:29.762482 25219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:29.762526 25219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:29.763278 25219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e535e21e9c48e99068577160e80f50" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37437 } }
I20260806 22:42:29.763415 25219 leader_election.cc:302] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93e535e21e9c48e99068577160e80f50; no voters:
I20260806 22:42:29.763571 25219 leader_election.cc:288] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:29.763626 25226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:29.763759 25226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [term 1 LEADER]: Becoming Leader. State: Replica: 93e535e21e9c48e99068577160e80f50, State: Running, Role: LEADER
I20260806 22:42:29.763813 25219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:29.763881 25226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e535e21e9c48e99068577160e80f50" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37437 } }
I20260806 22:42:29.764249 25226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93e535e21e9c48e99068577160e80f50. Latest consensus state: current_term: 1 leader_uuid: "93e535e21e9c48e99068577160e80f50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e535e21e9c48e99068577160e80f50" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37437 } } }
I20260806 22:42:29.764480 25226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:29.764407 25228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93e535e21e9c48e99068577160e80f50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e535e21e9c48e99068577160e80f50" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37437 } } }
I20260806 22:42:29.764905 25228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:29.764902 25235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:29.765642 25235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:29.767072 25235 catalog_manager.cc:1347] Generated new cluster ID: 9697eb9e3a2942a8bf4d5e1a2ce051f8
I20260806 22:42:29.767179 25235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:29.768108 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.762134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.19.74.254:37437)
0806 22:42:29.762383 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.762391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.762422 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:42:29.762536 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.762543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.762638 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.762783 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.762932 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.764698 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.765812 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.766419 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.766534 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.767414 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.767418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.767418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:29.767602 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.767648 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:29.767660 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.767720 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.767822 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.767912 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":126,"threads_started":1}
I20260806 22:42:29.769084 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:42:29.803059 25235 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:29.803591 25235 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:29.807574 25235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93e535e21e9c48e99068577160e80f50: Generated new TSK 0
I20260806 22:42:29.807773 25235 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:42:29.833904 25251 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 22:42:29.834053 25251 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:42:29.834259 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.813531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52508 (local address 127.19.74.254:37437)
0806 22:42:29.813616 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.813620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.813729 (+ 109us) server_negotiation.cc:407] Connection header received
0806 22:42:29.813819 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.813820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.813835 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.813912 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:42:29.814044 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.814510 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.815300 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.815495 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.815570 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.815758 (+ 188us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:42:29.833990 (+ 18232us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:42:29.834179 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52508: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 22:42:29.834339 25225 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52508: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 22:42:29.834498 25250 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:29.813510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37437 (local address 127.0.0.1:52508)
0806 22:42:29.813656 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.813667 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.813748 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.813911 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.813912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.813914 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:42:29.813972 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.813974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.814593 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.814595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.815208 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.815210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.815571 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.815572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.815637 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.815643 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:42:29.834170 (+ 18527us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:42:29.834313 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.74.254:37437: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260806 22:42:29.834815 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 25151
2026-08-06T22:42:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 22:42:29.875658 19755 mini_oidc.cc:152] Starting JWKS server
I20260806 22:42:29.876338 19755 webserver.cc:460] Webserver started at https://127.0.0.1:45993/ using document root <none> and password file <none>
I20260806 22:42:29.876410 19755 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:42:29.876597 19755 webserver.cc:460] Webserver started at http://127.0.0.1:45225/ using document root <none> and password file <none>
2026-08-06T22:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:29Z Disabled control of system clock
I20260806 22:42:29.885277 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:39285
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:39285
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45993/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:42:29.964684 25257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:29.964920 25257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:29.964956 25257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:29.966120 25257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:29.966190 25257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:29.966212 25257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:29.966244 25257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:29.966274 25257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:42:29.966297 25257 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:42:29.966321 25257 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45993/jwks.json
I20260806 22:42:29.967711 25257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:39285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:39285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45993/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:29.968003 25257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:29.968307 25257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:29.973239 25265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:29.973246 25262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.973353 25257 server_base.cc:1034] running on GCE node
W20260806 22:42:29.973279 25263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:29.973654 25257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:29.973882 25257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:29.975016 25257 hybrid_clock.cc:648] HybridClock initialized: now 1786056149974989 us; error 38 us; skew 500 ppm
I20260806 22:42:29.976209 25257 webserver.cc:460] Webserver started at http://127.19.74.254:43935/ using document root <none> and password file <none>
I20260806 22:42:29.976435 25257 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:29.976480 25257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:29.976658 25257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:29.977715 25257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "a9c59353c2e345e6a08437ad017c62f0"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "66e219240279fc85ac9689b9db418eed"
server_key_iv: "1a412b19f5117973a3a7aa992178e1c7"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.978128 25257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "a9c59353c2e345e6a08437ad017c62f0"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "66e219240279fc85ac9689b9db418eed"
server_key_iv: "1a412b19f5117973a3a7aa992178e1c7"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.979485 25257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:29.980213 25271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:29.980404 25257 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:29.980501 25257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "a9c59353c2e345e6a08437ad017c62f0"
format_stamp: "Formatted at 2026-08-06 22:42:29 on dist-test-slave-pgkr"
server_key: "66e219240279fc85ac9689b9db418eed"
server_key_iv: "1a412b19f5117973a3a7aa992178e1c7"
server_key_version: "encryptionkey@0"
I20260806 22:42:29.980573 25257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:29.995509 25257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:29.995805 25257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:29.995957 25257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:30.000044 25257 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:39285
I20260806 22:42:30.000097 25324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:39285 every 8 connection(s)
I20260806 22:42:30.000459 25257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:30.001125 25325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:30.003988 25325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0: Bootstrap starting.
I20260806 22:42:30.004546 25325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:30.004789 25325 log.cc:824] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:30.005409 25325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0: No bootstrap required, opened a new log
I20260806 22:42:30.006955 25325 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c59353c2e345e6a08437ad017c62f0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 39285 } }
I20260806 22:42:30.007091 25325 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:30.007115 25325 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c59353c2e345e6a08437ad017c62f0, State: Initialized, Role: FOLLOWER
I20260806 22:42:30.007216 25325 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c59353c2e345e6a08437ad017c62f0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 39285 } }
I20260806 22:42:30.007303 25325 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:30.007341 25325 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:30.007411 25325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:30.007993 25325 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c59353c2e345e6a08437ad017c62f0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 39285 } }
I20260806 22:42:30.008118 25325 leader_election.cc:302] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c59353c2e345e6a08437ad017c62f0; no voters:
I20260806 22:42:30.008311 25325 leader_election.cc:288] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:30.008353 25328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:30.008538 25328 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [term 1 LEADER]: Becoming Leader. State: Replica: a9c59353c2e345e6a08437ad017c62f0, State: Running, Role: LEADER
I20260806 22:42:30.008582 25325 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:30.008730 25328 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c59353c2e345e6a08437ad017c62f0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 39285 } }
I20260806 22:42:30.009114 25329 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9c59353c2e345e6a08437ad017c62f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c59353c2e345e6a08437ad017c62f0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 39285 } } }
I20260806 22:42:30.009262 25329 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:30.009281 25330 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c59353c2e345e6a08437ad017c62f0. Latest consensus state: current_term: 1 leader_uuid: "a9c59353c2e345e6a08437ad017c62f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c59353c2e345e6a08437ad017c62f0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 39285 } } }
I20260806 22:42:30.009374 25330 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:30.009449 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 25257
I20260806 22:42:30.009569 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:30.009673 25337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:30.009771 19755 external_mini_cluster.cc:1428] Setting key 4cc8330e2853d6af86bca393f16ba4c7
I20260806 22:42:30.010213 25337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:30.011957 25337 catalog_manager.cc:1347] Generated new cluster ID: 1fc43e27453d4f0f9b7a1ed802b038bf
I20260806 22:42:30.012012 25337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:30.015385 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:30.010433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46294 (local address 127.19.74.254:39285)
0806 22:42:30.010646 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:30.010656 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:30.010690 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:42:30.010868 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:30.010877 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:30.010952 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:30.011389 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:30.011498 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.012570 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.013366 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.013821 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.013961 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:30.014652 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:30.014656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:30.014656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:30.014846 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:30.014879 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:30.014890 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:30.014961 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:30.015049 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:42:30.015224 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260806 22:42:30.016230 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:42:30.031668 25337 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:30.032176 25337 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:30.040676 25337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9c59353c2e345e6a08437ad017c62f0: Generated new TSK 0
I20260806 22:42:30.040891 25337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:42:30.049654 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:30.046296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.19.74.254:39285)
0806 22:42:30.046357 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:30.046359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:30.046441 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:42:30.046535 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:30.046536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:30.046559 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:30.046684 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:42:30.046841 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.047348 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.048097 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.048307 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.048380 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:30.048569 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:42:30.049442 (+ 873us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45993/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:42:30.049570 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46306: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45993/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 22:42:30.049708 25345 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46306: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45993/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 22:42:30.049762 25355 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:30.046201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:39285 (local address 127.0.0.1:46306)
0806 22:42:30.046365 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:30.046376 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:30.046461 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:30.046687 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:30.046688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:30.046690 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:42:30.046749 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:30.046751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.047432 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.047433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:30.047998 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:30.048000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.048381 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.048382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:30.048440 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:30.048445 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:42:30.049544 (+ 1099us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45993/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:42:30.049659 (+ 115us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.74.254:39285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45993/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260806 22:42:30.050010 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 25257
E20260806 22:42:30.062152 25356 webserver.cc:533] Webserver: sslize failed:
E20260806 22:42:30.062220 25356 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T22:42:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (215 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T22:42:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:30Z Disabled control of system clock
I20260806 22:42:30.099427 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:43335
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:43335
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/testchainca.cert with env {}
W20260806 22:42:30.179948 25360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:30.180131 25360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:30.180152 25360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:30.181185 25360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:30.181216 25360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:30.181228 25360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:30.181241 25360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:30.181252 25360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:42:30.181263 25360 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:42:30.181274 25360 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 22:42:30.182736 25360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:43335
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:43335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:30.182920 25360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:30.183070 25360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:30.187844 25368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:30.187831 25366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:30.187819 25365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:30.188036 25360 server_base.cc:1034] running on GCE node
I20260806 22:42:30.188355 25360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:30.188668 25360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:30.189837 25360 hybrid_clock.cc:648] HybridClock initialized: now 1786056150189825 us; error 43 us; skew 500 ppm
I20260806 22:42:30.191044 25360 webserver.cc:460] Webserver started at http://127.19.74.254:42011/ using document root <none> and password file <none>
I20260806 22:42:30.191282 25360 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:30.191354 25360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:30.191499 25360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:30.192603 25360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca"
format_stamp: "Formatted at 2026-08-06 22:42:30 on dist-test-slave-pgkr"
server_key: "d496195fee56a0bd224252e9f799e4f3"
server_key_iv: "ec7ec6c0146349894a4e919e54bb4d9d"
server_key_version: "encryptionkey@0"
I20260806 22:42:30.192960 25360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca"
format_stamp: "Formatted at 2026-08-06 22:42:30 on dist-test-slave-pgkr"
server_key: "d496195fee56a0bd224252e9f799e4f3"
server_key_iv: "ec7ec6c0146349894a4e919e54bb4d9d"
server_key_version: "encryptionkey@0"
I20260806 22:42:30.194255 25360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:30.195052 25375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:30.195228 25360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:30.195338 25360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca"
format_stamp: "Formatted at 2026-08-06 22:42:30 on dist-test-slave-pgkr"
server_key: "d496195fee56a0bd224252e9f799e4f3"
server_key_iv: "ec7ec6c0146349894a4e919e54bb4d9d"
server_key_version: "encryptionkey@0"
I20260806 22:42:30.195413 25360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:30.225029 25360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:30.225378 25360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:30.225540 25360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:30.229799 25360 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:43335
I20260806 22:42:30.229846 25428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:43335 every 8 connection(s)
I20260806 22:42:30.230216 25360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:30.231312 25429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:30.233649 25429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca: Bootstrap starting.
I20260806 22:42:30.233938 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 25360
I20260806 22:42:30.234048 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:30.234239 19755 external_mini_cluster.cc:1428] Setting key febc3375c47c8a97086878c3ddb3ced9
I20260806 22:42:30.234404 25429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:30.234792 25429 log.cc:824] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:30.235538 25429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca: No bootstrap required, opened a new log
I20260806 22:42:30.237730 25429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43335 } }
I20260806 22:42:30.237897 25429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:30.237928 25429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d0a5b47d6424de3bcd3d69cbacb5aca, State: Initialized, Role: FOLLOWER
I20260806 22:42:30.238044 25429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43335 } }
I20260806 22:42:30.238154 25429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:30.238204 25429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:30.238256 25429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:30.238922 25429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43335 } }
I20260806 22:42:30.239063 25429 leader_election.cc:302] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d0a5b47d6424de3bcd3d69cbacb5aca; no voters:
I20260806 22:42:30.239265 25429 leader_election.cc:288] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:30.239384 25434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:30.239540 25429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:30.239591 25434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [term 1 LEADER]: Becoming Leader. State: Replica: 1d0a5b47d6424de3bcd3d69cbacb5aca, State: Running, Role: LEADER
I20260806 22:42:30.239753 25434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43335 } }
I20260806 22:42:30.240145 25435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43335 } } }
I20260806 22:42:30.240155 25436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d0a5b47d6424de3bcd3d69cbacb5aca. Latest consensus state: current_term: 1 leader_uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d0a5b47d6424de3bcd3d69cbacb5aca" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43335 } } }
I20260806 22:42:30.240301 25435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:30.240309 25436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:30.240545 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:30.234800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.19.74.254:43335)
0806 22:42:30.235023 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:30.235032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:30.235069 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:42:30.235594 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:30.235602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:30.235701 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:30.235832 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:30.236095 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.236973 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.238115 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.238643 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.238752 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:30.239672 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:30.239676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:30.239677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:42:30.239934 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:30.239981 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:42:30.239995 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:30.240037 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:30.240219 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 22:42:30.240313 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260806 22:42:30.240586 25443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:30.241225 25443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:30.242034 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:42:30.242897 25443 catalog_manager.cc:1347] Generated new cluster ID: 7bdf2d13362146ce81184677a542e41a
I20260806 22:42:30.242993 25443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:30.252169 25443 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:30.252827 25443 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:30.257805 25443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d0a5b47d6424de3bcd3d69cbacb5aca: Generated new TSK 0
I20260806 22:42:30.257990 25443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:42:30.387102 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:30.262743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38366 (local address 127.19.74.254:43335)
0806 22:42:30.262778 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:30.262781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:30.262898 (+ 117us) server_negotiation.cc:407] Connection header received
0806 22:42:30.262980 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:30.262980 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:30.262991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:30.263065 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:42:30.263220 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.263689 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.264433 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.264641 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.264713 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:30.264878 (+ 165us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:42:30.386772 (+121894us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:42:30.387011 (+ 239us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:42:30.387209 25432 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 22:42:30.387205 25458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:30.262695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:43335 (local address 127.0.0.1:38366)
0806 22:42:30.262834 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:30.262843 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:30.262915 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:30.263079 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:30.263080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:30.263082 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:42:30.263146 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:30.263148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.263783 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.263784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:30.264348 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:30.264350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:30.264714 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:30.264715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:30.264773 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:30.264777 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:42:30.386939 (+122162us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:42:30.387075 (+ 136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.74.254:43335: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260806 22:42:30.387567 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 25360
2026-08-06T22:42:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (316 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:30 dist-test-slave-pgkr krb5kdc[25473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:30 dist-test-slave-pgkr krb5kdc[25473](info): set up 2 sockets
Aug 06 22:42:30 dist-test-slave-pgkr krb5kdc[25473](info): commencing operation
krb5kdc: starting...
W20260806 22:42:31.637198 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:31 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:31.757582 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:45261
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:45261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:31.839129 25486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:31.839375 25486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:31.839414 25486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:31.840584 25486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:31.840659 25486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:31.840683 25486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:31.840713 25486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:31.840762 25486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:31.842273 25486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:45261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:45261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:31.842563 25486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:31.842905 25486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:31.847819 25491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:31.847844 25492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:31.847880 25486 server_base.cc:1034] running on GCE node
W20260806 22:42:31.847852 25495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:31.848346 25486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:31.848640 25486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:31.849807 25486 hybrid_clock.cc:648] HybridClock initialized: now 1786056151849801 us; error 35 us; skew 500 ppm
Aug 06 22:42:31 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:31.852385 25486 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:31.852854 25486 webserver.cc:460] Webserver started at http://127.19.74.254:41725/ using document root <none> and password file <none>
I20260806 22:42:31.853116 25486 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:31.853185 25486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:31.853343 25486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:31.854364 25486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "fecbb9279a7a44148f80d78b0e2fa01b"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-pgkr"
server_key: "7a5369bd43792de90958d1b884fb9c04"
server_key_iv: "925e321e255ae6ac8831d508c9e84351"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.854748 25486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "fecbb9279a7a44148f80d78b0e2fa01b"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-pgkr"
server_key: "7a5369bd43792de90958d1b884fb9c04"
server_key_iv: "925e321e255ae6ac8831d508c9e84351"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.856041 25486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:31.856746 25502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:31.856922 25486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:31.857028 25486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "fecbb9279a7a44148f80d78b0e2fa01b"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-pgkr"
server_key: "7a5369bd43792de90958d1b884fb9c04"
server_key_iv: "925e321e255ae6ac8831d508c9e84351"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.857115 25486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:31.872808 25486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:31.875059 25486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:31.875229 25486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:31.879354 25486 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:45261
I20260806 22:42:31.879388 25556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:45261 every 8 connection(s)
I20260806 22:42:31.879753 25486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:31.880635 25557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:31.881901 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 25486
I20260806 22:42:31.882025 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:31.882230 19755 external_mini_cluster.cc:1428] Setting key 50794397695307c32372fb92aed1b62e
I20260806 22:42:31.883533 25557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b: Bootstrap starting.
I20260806 22:42:31.884249 25557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:31.884589 25557 log.cc:824] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:31.885349 25557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b: No bootstrap required, opened a new log
I20260806 22:42:31.887691 25557 raft_consensus.cc:348] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecbb9279a7a44148f80d78b0e2fa01b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45261 } }
I20260806 22:42:31.887826 25557 raft_consensus.cc:374] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:31.887849 25557 raft_consensus.cc:729] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecbb9279a7a44148f80d78b0e2fa01b, State: Initialized, Role: FOLLOWER
I20260806 22:42:31.888193 25557 consensus_queue.cc:260] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecbb9279a7a44148f80d78b0e2fa01b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45261 } }
I20260806 22:42:31.888320 25557 raft_consensus.cc:388] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:31.888370 25557 raft_consensus.cc:482] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:31.888424 25557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:31.889026 25557 raft_consensus.cc:504] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecbb9279a7a44148f80d78b0e2fa01b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45261 } }
I20260806 22:42:31.889144 25557 leader_election.cc:302] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fecbb9279a7a44148f80d78b0e2fa01b; no voters:
I20260806 22:42:31.889328 25557 leader_election.cc:288] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:31.889386 25562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:31.889568 25562 raft_consensus.cc:686] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [term 1 LEADER]: Becoming Leader. State: Replica: fecbb9279a7a44148f80d78b0e2fa01b, State: Running, Role: LEADER
I20260806 22:42:31.889593 25557 sys_catalog.cc:564] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:31.889709 25562 consensus_queue.cc:237] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecbb9279a7a44148f80d78b0e2fa01b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45261 } }
Aug 06 22:42:31 dist-test-slave-pgkr krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:31.890602 25563 sys_catalog.cc:455] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fecbb9279a7a44148f80d78b0e2fa01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecbb9279a7a44148f80d78b0e2fa01b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45261 } } }
I20260806 22:42:31.890717 25563 sys_catalog.cc:458] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:31.890940 25574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:31.890955 25564 sys_catalog.cc:455] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fecbb9279a7a44148f80d78b0e2fa01b. Latest consensus state: current_term: 1 leader_uuid: "fecbb9279a7a44148f80d78b0e2fa01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fecbb9279a7a44148f80d78b0e2fa01b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45261 } } }
I20260806 22:42:31.891284 25564 sys_catalog.cc:458] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:31.891496 25574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:31.893886 25574 catalog_manager.cc:1347] Generated new cluster ID: e6220ea73124455e88d7e55a5aa33af3
I20260806 22:42:31.893947 25574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:31.894212 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.882856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55850 (local address 127.19.74.254:45261)
0806 22:42:31.883065 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.883077 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.883115 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:42:31.883687 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.883696 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.883759 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.883925 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.884583 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.885967 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.886688 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.887383 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.887499 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.890394 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.890398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.890398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.890410 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.892438 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.892836 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.892838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.892838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.892917 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.893147 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.893148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.893148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.893378 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.893484 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.893871 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.894007 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
I20260806 22:42:31.912228 25574 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:31.912781 25574 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:31.917644 25574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fecbb9279a7a44148f80d78b0e2fa01b: Generated new TSK 0
I20260806 22:42:31.917826 25574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:31.946087 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45261
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:32.028875 25584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:32.029079 25584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:32.029109 25584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:32.030244 25584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:32.030277 25584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:32.030303 25584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:32.031807 25584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:32.031992 25584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:32.032176 25584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:32.037209 25592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:32.037215 25590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:32.037457 25584 server_base.cc:1034] running on GCE node
W20260806 22:42:32.037230 25589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:32.037690 25584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:32.037889 25584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:32.039050 25584 hybrid_clock.cc:648] HybridClock initialized: now 1786056152039053 us; error 47 us; skew 500 ppm
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.041632 25584 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:32.042052 25584 webserver.cc:460] Webserver started at http://127.19.74.193:34275/ using document root <none> and password file <none>
I20260806 22:42:32.042301 25584 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:32.042371 25584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:32.042529 25584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:32.043681 25584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "efde699ff5aa44269d45acd1e5635b8c"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "5a7c2fbe4a013498b267235b326ace1f"
server_key_iv: "7c3e84c2ae942caf8e1c2a85ecf6ee04"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.044047 25584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "efde699ff5aa44269d45acd1e5635b8c"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "5a7c2fbe4a013498b267235b326ace1f"
server_key_iv: "7c3e84c2ae942caf8e1c2a85ecf6ee04"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.045374 25584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:32.046105 25599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.046283 25584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:32.046375 25584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "efde699ff5aa44269d45acd1e5635b8c"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "5a7c2fbe4a013498b267235b326ace1f"
server_key_iv: "7c3e84c2ae942caf8e1c2a85ecf6ee04"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.046455 25584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:32.055706 25584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:32.057440 25584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:32.057591 25584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:32.057814 25584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:32.058115 25584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:32.058167 25584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.058215 25584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:32.058243 25584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.064213 25584 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:37191
I20260806 22:42:32.064280 25716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:37191 every 8 connection(s)
I20260806 22:42:32.064625 25584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:32.071075 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 25584
I20260806 22:42:32.071200 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:32.071422 19755 external_mini_cluster.cc:1428] Setting key 70560594602b1eb2984d09711840e435
I20260806 22:42:32.074276 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.065346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:58855 (local address 127.19.74.254:45261)
0806 22:42:32.065424 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.065427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.065614 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:42:32.066569 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.066570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.066583 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.066684 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.067306 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.067811 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.068777 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.069003 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.069085 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.071280 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:32.071281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:32.071282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:32.071291 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:32.072781 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.073275 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.073276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.073276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.073329 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.073599 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.073599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.073599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.073696 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:32.073781 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:32.074068 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:42:32.074166 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:42:32.074409 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.065308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45261 (local address 127.19.74.193:58855)
0806 22:42:32.065492 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:32.065527 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:32.066453 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:32.066763 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:32.066767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:32.067011 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:32.067197 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:32.067200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.067907 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.067909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:32.068674 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:32.068675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.069116 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.069117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:32.069226 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.069650 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:32.069658 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:32.071127 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:32.072861 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:32.072869 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:32.072871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:32.073200 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:32.073391 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:32.073392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:32.073392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:32.073536 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:32.073777 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:32.073785 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:32.073922 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 22:42:32.074027 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:42:32.074183 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260806 22:42:32.074896 25718 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45261
I20260806 22:42:32.075001 25718 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:32.075201 25718 heartbeater.cc:511] Master 127.19.74.254:45261 requested a full tablet report, sending...
I20260806 22:42:32.075723 25519 ts_manager.cc:194] Registered new tserver with Master: efde699ff5aa44269d45acd1e5635b8c (127.19.74.193:37191)
I20260806 22:42:32.076352 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:58855
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:32.097424 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45261
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:32.178965 25723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:32.179163 25723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:32.179186 25723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:32.180266 25723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:32.180299 25723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:32.180325 25723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:32.181695 25723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:32.181872 25723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:32.182035 25723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:32.187052 25728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:32.187052 25731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:32.187080 25729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:32.187376 25723 server_base.cc:1034] running on GCE node
I20260806 22:42:32.187674 25723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:32.187932 25723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:32.189103 25723 hybrid_clock.cc:648] HybridClock initialized: now 1786056152189100 us; error 37 us; skew 500 ppm
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.191916 25723 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:32.192374 25723 webserver.cc:460] Webserver started at http://127.19.74.194:42437/ using document root <none> and password file <none>
I20260806 22:42:32.192560 25723 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:32.192602 25723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:32.192691 25723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:32.193632 25723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "c12544e53c2a4963b046f8fba295089d"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "3310fbc9fda2193e9f9c8c656d0e7f81"
server_key_iv: "6897d13d730e121f3eba4633b9e85626"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.193921 25723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "c12544e53c2a4963b046f8fba295089d"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "3310fbc9fda2193e9f9c8c656d0e7f81"
server_key_iv: "6897d13d730e121f3eba4633b9e85626"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.195274 25723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:32.195974 25738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.196153 25723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:32.196239 25723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "c12544e53c2a4963b046f8fba295089d"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "3310fbc9fda2193e9f9c8c656d0e7f81"
server_key_iv: "6897d13d730e121f3eba4633b9e85626"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.196290 25723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:32.216126 25723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:32.218464 25723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:32.218578 25723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:32.218793 25723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:32.219121 25723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:32.219157 25723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.219182 25723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:32.219225 25723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.224822 25723 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:46113
I20260806 22:42:32.224925 25854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:46113 every 8 connection(s)
I20260806 22:42:32.225260 25723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:32.232559 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 25723
I20260806 22:42:32.232681 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:32.232898 19755 external_mini_cluster.cc:1428] Setting key 193ad1e3d7883314b5b6a64f472455ab
I20260806 22:42:32.234800 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.226057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:38501 (local address 127.19.74.254:45261)
0806 22:42:32.226125 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.226127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.226338 (+ 211us) server_negotiation.cc:407] Connection header received
0806 22:42:32.227362 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.227363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.227372 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.227449 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.228046 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.228550 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.229481 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.229661 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.229710 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.231727 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:32.231729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:32.231729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:32.231736 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:32.233335 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.233848 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.233849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.233850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.233910 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.234134 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.234134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.234135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.234284 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:32.234351 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:32.234558 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 22:42:32.234677 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:42:32.234863 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.226034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45261 (local address 127.19.74.194:38501)
0806 22:42:32.226213 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:32.226254 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:32.227275 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:32.227506 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:32.227509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:32.227764 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:32.227967 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:32.227971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.228631 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.228632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:32.229409 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:32.229412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.229731 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.229731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:32.229853 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.230297 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:32.230305 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:32.231649 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:32.233411 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:32.233418 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:32.233420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:32.233779 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:32.233978 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:32.233979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:32.233979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:32.234073 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:32.234344 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:32.234348 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:32.234446 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 22:42:32.234520 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:42:32.234688 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260806 22:42:32.235474 25855 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45261
I20260806 22:42:32.235592 25855 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:32.235785 25855 heartbeater.cc:511] Master 127.19.74.254:45261 requested a full tablet report, sending...
I20260806 22:42:32.236186 25519 ts_manager.cc:194] Registered new tserver with Master: c12544e53c2a4963b046f8fba295089d (127.19.74.194:46113)
I20260806 22:42:32.236691 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:38501
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:32.259333 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45261
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:32.341672 25860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:32.341918 25860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:32.341952 25860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:32.343048 25860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:32.343113 25860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:32.343154 25860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:32.344691 25860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:32.344951 25860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:32.345177 25860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:32.350205 25866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:32.350358 25860 server_base.cc:1034] running on GCE node
W20260806 22:42:32.350266 25865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:32.350221 25868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:32.350767 25860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:32.351006 25860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:32.352149 25860 hybrid_clock.cc:648] HybridClock initialized: now 1786056152352139 us; error 26 us; skew 500 ppm
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.354735 25860 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:32.355193 25860 webserver.cc:460] Webserver started at http://127.19.74.195:41199/ using document root <none> and password file <none>
I20260806 22:42:32.355437 25860 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:32.355505 25860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:32.355651 25860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:32.356643 25860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "2fd5bb568c8144a69574d05beefd3419"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "6825221ccf9248206a0bc1baa4ee1019"
server_key_iv: "cb42e170a104cb40d35b3d8e3cbe2bfa"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.357003 25860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "2fd5bb568c8144a69574d05beefd3419"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "6825221ccf9248206a0bc1baa4ee1019"
server_key_iv: "cb42e170a104cb40d35b3d8e3cbe2bfa"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.358312 25860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:32.359148 25875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.359354 25860 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:42:32.359453 25860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "2fd5bb568c8144a69574d05beefd3419"
format_stamp: "Formatted at 2026-08-06 22:42:32 on dist-test-slave-pgkr"
server_key: "6825221ccf9248206a0bc1baa4ee1019"
server_key_iv: "cb42e170a104cb40d35b3d8e3cbe2bfa"
server_key_version: "encryptionkey@0"
I20260806 22:42:32.359526 25860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:32.377003 25860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:32.379462 25860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:32.379631 25860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:32.379873 25860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:32.380208 25860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:32.380242 25860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.380283 25860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:32.380312 25860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:32.386271 25860 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:43455
I20260806 22:42:32.386350 25994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:43455 every 8 connection(s)
I20260806 22:42:32.386734 25860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:32.395002 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 25860
I20260806 22:42:32.395079 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:32.395256 19755 external_mini_cluster.cc:1428] Setting key 420f0836e5b8620a4021eb908ec43a33
I20260806 22:42:32.396435 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.387463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:40261 (local address 127.19.74.254:45261)
0806 22:42:32.387546 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.387548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.387713 (+ 165us) server_negotiation.cc:407] Connection header received
0806 22:42:32.388601 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.388602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.388621 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.388673 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.389317 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.389798 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.391011 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.391228 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.391324 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.393470 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:32.393472 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:32.393472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:32.393480 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:32.394828 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.395305 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.395306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.395306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.395372 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.395743 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.395745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.395745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.395906 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:32.395978 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:32.396300 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 22:42:32.396371 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:42:32.396598 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.387423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45261 (local address 127.19.74.195:40261)
0806 22:42:32.387591 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:32.387630 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:32.388515 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:32.388726 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:32.388730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:32.388967 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:32.389245 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:32.389250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.389894 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.389895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:32.390914 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:32.390917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.391335 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.391337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:32.391479 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.391999 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:32.392008 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:32.393363 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:32.394922 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:32.394929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:32.394931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:32.395227 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:32.395522 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:32.395522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:32.395523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:32.395671 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:32.395987 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:32.395994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:32.396157 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 22:42:32.396271 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:42:32.396397 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
I20260806 22:42:32.397032 25995 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45261
I20260806 22:42:32.397156 25995 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:32.397352 25995 heartbeater.cc:511] Master 127.19.74.254:45261 requested a full tablet report, sending...
I20260806 22:42:32.397716 25519 ts_manager.cc:194] Registered new tserver with Master: 2fd5bb568c8144a69574d05beefd3419 (127.19.74.195:43455)
I20260806 22:42:32.398104 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:40261
I20260806 22:42:32.407011 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/test-admin.keytab.
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.422973 19755 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 22:42:32.424575 26002 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:32.423813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.19.74.193:37191)
0806 22:42:32.423967 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.423971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.423981 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:32.424006 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.424008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.424021 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.424089 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.424468 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58058: BlockingRecv error: recv got EOF from 127.0.0.1:58058 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
W20260806 22:42:32.424674 19755 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.425639 19755 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.426514 19755 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:32.433753 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.427029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.19.74.193:37191)
0806 22:42:32.427090 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.427092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.427102 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:32.427142 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.427143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.427152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.427198 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.427563 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.428316 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.428984 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.429285 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.429338 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.431331 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:32.431339 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:32.431339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:32.431345 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:32.432711 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.433012 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.433014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.433015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.433067 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.433230 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.433231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.433231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.433347 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:32.433453 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:32.433520 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 22:42:32.433576 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 22:42:32.435503 26002 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:32.435004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.19.74.193:37191)
0806 22:42:32.435041 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.435042 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.435081 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:42:32.435146 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.435147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.435155 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.435199 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.435437 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58074: BlockingRecv error: recv got EOF from 127.0.0.1:58074 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:42:32.435606 19755 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.436514 19755 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.437386 19755 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:32.443944 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.437837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.19.74.193:37191)
0806 22:42:32.437877 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.437879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.437890 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:32.437962 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.437963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.437972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.438025 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.438394 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.438941 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.439587 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.439784 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.439857 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.441656 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:32.441657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:32.441658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:32.441664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:32.442815 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.443158 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.443160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.443160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.443216 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.443414 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.443414 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.443415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.443522 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:32.443619 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:32.443733 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:42:32.443803 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:42:32.445510 26002 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:32.444874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58084 (local address 127.19.74.193:37191)
0806 22:42:32.444910 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.444912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.444958 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:42:32.445026 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.445027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.445035 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.445087 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.445467 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58084: BlockingRecv error: recv got EOF from 127.0.0.1:58084 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:42:32.445578 19755 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056127658350-19755-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.446475 19755 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:32.447352 19755 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.193@KRBTEST.COM
I20260806 22:42:32.453493 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.447806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.19.74.193:37191)
0806 22:42:32.447837 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.447839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.447847 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:32.447869 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.447869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.447877 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.447929 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.448247 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.448675 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.449265 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.449425 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.449474 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.451402 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:32.451403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:32.451404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:32.451409 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:32.452644 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.452936 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.452937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.452937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.452984 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.453140 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.453141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.453141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.453218 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:32.453282 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:32.453380 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:42:32.453445 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:42:32.454082 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 25584
I20260806 22:42:32.461395 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 25723
I20260806 22:42:32.466529 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 25860
I20260806 22:42:32.472216 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 25486
2026-08-06T22:42:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1903 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[26016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[26016](info): set up 2 sockets
Aug 06 22:42:32 dist-test-slave-pgkr krb5kdc[26016](info): commencing operation
krb5kdc: starting...
W20260806 22:42:33.520560 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:33 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.254.keytab.
Entry for principal oryx/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.254.keytab.
I20260806 22:42:33.635517 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42469
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.254.keytab
--principal=oryx/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:42469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:33.717201 26030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:33.717438 26030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:33.717475 26030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:33.718688 26030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:33.718755 26030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:33.718775 26030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:33.718803 26030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:33.718864 26030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:33.720391 26030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:42469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.254.keytab
--principal=oryx/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:33.720671 26030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:33.720901 26030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:33.725863 26038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:33.725909 26030 server_base.cc:1034] running on GCE node
W20260806 22:42:33.725878 26036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:33.725873 26035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:33.726259 26030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:33.726563 26030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:33.727756 26030 hybrid_clock.cc:648] HybridClock initialized: now 1786056153727744 us; error 29 us; skew 500 ppm
Aug 06 22:42:33 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:33.730209 26030 init.cc:377] Logged in from keytab as oryx/127.19.74.254@KRBTEST.COM (short username oryx)
I20260806 22:42:33.730693 26030 webserver.cc:460] Webserver started at http://127.19.74.254:41989/ using document root <none> and password file <none>
I20260806 22:42:33.730921 26030 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:33.730988 26030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:33.731146 26030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:33.732182 26030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "9854708559ab44a9a626401c6b3a2499"
format_stamp: "Formatted at 2026-08-06 22:42:33 on dist-test-slave-pgkr"
server_key: "ac35237efde8689fdf34fadfc258020b"
server_key_iv: "bdbfd3c41e926b17286c22edbda02bce"
server_key_version: "encryptionkey@0"
I20260806 22:42:33.732583 26030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "9854708559ab44a9a626401c6b3a2499"
format_stamp: "Formatted at 2026-08-06 22:42:33 on dist-test-slave-pgkr"
server_key: "ac35237efde8689fdf34fadfc258020b"
server_key_iv: "bdbfd3c41e926b17286c22edbda02bce"
server_key_version: "encryptionkey@0"
I20260806 22:42:33.733922 26030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:33.734645 26045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:33.734826 26030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:33.734929 26030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "9854708559ab44a9a626401c6b3a2499"
format_stamp: "Formatted at 2026-08-06 22:42:33 on dist-test-slave-pgkr"
server_key: "ac35237efde8689fdf34fadfc258020b"
server_key_iv: "bdbfd3c41e926b17286c22edbda02bce"
server_key_version: "encryptionkey@0"
I20260806 22:42:33.735008 26030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:33.752096 26030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:33.754019 26030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:33.754195 26030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:33.758203 26030 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:42469
I20260806 22:42:33.758244 26099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:42469 every 8 connection(s)
I20260806 22:42:33.758611 26030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:33.759752 26100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:33.760635 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 26030
I20260806 22:42:33.760742 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:33.760962 19755 external_mini_cluster.cc:1428] Setting key 861f0954d7c242b5f51ed0f5e8722821
I20260806 22:42:33.763079 26100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: Bootstrap starting.
I20260806 22:42:33.763746 26100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:33.764045 26100 log.cc:824] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:33.764740 26100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: No bootstrap required, opened a new log
I20260806 22:42:33.766762 26100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } }
I20260806 22:42:33.766963 26100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:33.767002 26100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9854708559ab44a9a626401c6b3a2499, State: Initialized, Role: FOLLOWER
I20260806 22:42:33.767143 26100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } }
I20260806 22:42:33.767251 26100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:33.767287 26100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:33.767336 26100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 22:42:33 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:33.768023 26100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } }
I20260806 22:42:33.768132 26100 leader_election.cc:302] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9854708559ab44a9a626401c6b3a2499; no voters:
I20260806 22:42:33.768311 26100 leader_election.cc:288] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:33.768371 26105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:33.768539 26105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 1 LEADER]: Becoming Leader. State: Replica: 9854708559ab44a9a626401c6b3a2499, State: Running, Role: LEADER
I20260806 22:42:33.768590 26100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:33.768615 26105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } }
I20260806 22:42:33.768929 26105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9854708559ab44a9a626401c6b3a2499. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } }
I20260806 22:42:33.769011 26105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:33.769034 26106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } }
I20260806 22:42:33.769088 26106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:33.769856 26111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:33.770433 26111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:33.772056 26111 catalog_manager.cc:1347] Generated new cluster ID: dc5f99a5646847c789e5f4e81ec45ea9
I20260806 22:42:33.772127 26111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:33.772131 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:33.761502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58546 (local address 127.19.74.254:42469)
0806 22:42:33.761681 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:33.761694 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:33.761719 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:42:33.762128 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:33.762134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:33.762180 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:33.762290 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:33.763132 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:33.764100 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:33.764895 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:33.765297 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:33.765420 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:33.768045 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:33.768049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:33.768049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:33.768061 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:33.770201 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:33.770644 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:33.770648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:33.770648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:33.770746 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:33.770963 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:33.770964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:33.770964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:33.771277 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:33.771415 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:33.771801 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 22:42:33.771943 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":102,"threads_started":1}
I20260806 22:42:33.781071 26111 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:33.781555 26111 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:33.792188 26111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: Generated new TSK 0
I20260806 22:42:33.792371 26111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab.
Entry for principal oryx/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab.
I20260806 22:42:33.829268 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42469
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab
--principal=oryx/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:33.917074 26126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:33.917302 26126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:33.917344 26126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:33.918483 26126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:33.918551 26126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:33.918596 26126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:33.920039 26126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab
--principal=oryx/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:33.920292 26126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:33.920512 26126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:33.925539 26132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:33.925587 26131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:33.925773 26126 server_base.cc:1034] running on GCE node
W20260806 22:42:33.925786 26134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:33.926013 26126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:33.926330 26126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:33.927528 26126 hybrid_clock.cc:648] HybridClock initialized: now 1786056153927514 us; error 47 us; skew 500 ppm
Aug 06 22:42:33 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:33.930033 26126 init.cc:377] Logged in from keytab as oryx/127.19.74.193@KRBTEST.COM (short username oryx)
I20260806 22:42:33.930509 26126 webserver.cc:460] Webserver started at http://127.19.74.193:39691/ using document root <none> and password file <none>
I20260806 22:42:33.930776 26126 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:33.930846 26126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:33.930984 26126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:33.931984 26126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "0df6c1f5bac54740a8d25e6118c6cdac"
format_stamp: "Formatted at 2026-08-06 22:42:33 on dist-test-slave-pgkr"
server_key: "7d9d29cfb5e325780ce54716d0c90be2"
server_key_iv: "d693e3a8ff17be5b86079bc9e52d965f"
server_key_version: "encryptionkey@0"
I20260806 22:42:33.932367 26126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "0df6c1f5bac54740a8d25e6118c6cdac"
format_stamp: "Formatted at 2026-08-06 22:42:33 on dist-test-slave-pgkr"
server_key: "7d9d29cfb5e325780ce54716d0c90be2"
server_key_iv: "d693e3a8ff17be5b86079bc9e52d965f"
server_key_version: "encryptionkey@0"
I20260806 22:42:33.933670 26126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:33.934384 26141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:33.934553 26126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:33.934679 26126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "0df6c1f5bac54740a8d25e6118c6cdac"
format_stamp: "Formatted at 2026-08-06 22:42:33 on dist-test-slave-pgkr"
server_key: "7d9d29cfb5e325780ce54716d0c90be2"
server_key_iv: "d693e3a8ff17be5b86079bc9e52d965f"
server_key_version: "encryptionkey@0"
I20260806 22:42:33.934754 26126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:33.943439 26126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:33.945138 26126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:33.945292 26126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:33.945508 26126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:33.945785 26126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:33.945865 26126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:33.945896 26126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:33.945925 26126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:33.951692 26126 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:37361
I20260806 22:42:33.951727 26258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:37361 every 8 connection(s)
I20260806 22:42:33.952025 26126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:33.955338 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 26126
I20260806 22:42:33.955435 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:33.955622 19755 external_mini_cluster.cc:1428] Setting key 57b703e59fc90f5226cf6d3cfae321c8
Aug 06 22:42:33 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:33.961987 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:33.952759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:45967 (local address 127.19.74.254:42469)
0806 22:42:33.952826 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:33.952828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:33.953020 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:42:33.953455 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:33.953456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:33.953466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:33.953523 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:33.954425 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:33.954997 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:33.956263 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:33.956594 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:33.956669 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:33.959106 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:33.959108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:33.959108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:33.959117 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:33.960471 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:33.960997 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:33.960998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:33.960998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:33.961062 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:33.961357 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:33.961358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:33.961358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:33.961459 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:33.961511 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:33.961849 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 22:42:33.961924 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:42:33.962309 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:33.952723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:45967)
0806 22:42:33.952898 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:33.952938 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:33.953376 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:33.953613 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:33.953617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:33.954161 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:33.954354 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:33.954357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:33.955074 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:33.955074 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:33.956165 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:33.956168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:33.956685 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:33.956686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:33.956791 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:33.957272 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:33.957281 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:33.958999 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:33.960556 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:33.960565 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:33.960568 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:33.960911 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:33.961137 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:33.961138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:33.961139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:33.961290 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:33.961525 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:33.961531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:33.961701 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:42:33.961830 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 22:42:33.962013 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260806 22:42:33.962918 26259 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42469
I20260806 22:42:33.963013 26259 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:33.963212 26259 heartbeater.cc:511] Master 127.19.74.254:42469 requested a full tablet report, sending...
I20260806 22:42:33.963732 26062 ts_manager.cc:194] Registered new tserver with Master: 0df6c1f5bac54740a8d25e6118c6cdac (127.19.74.193:37361)
I20260806 22:42:33.964319 26062 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:45967
WARNING: no policy specified for oryx/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab.
Entry for principal oryx/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab.
I20260806 22:42:33.982009 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42469
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab
--principal=oryx/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:34.062719 26264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:34.062945 26264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:34.062983 26264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:34.064117 26264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:34.064182 26264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:34.064224 26264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:34.065603 26264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab
--principal=oryx/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:34.065861 26264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:34.066112 26264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:34.071368 26272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.071362 26269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.071364 26270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.071601 26264 server_base.cc:1034] running on GCE node
I20260806 22:42:34.071805 26264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:34.072045 26264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:34.073202 26264 hybrid_clock.cc:648] HybridClock initialized: now 1786056154073194 us; error 42 us; skew 500 ppm
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:34.075801 26264 init.cc:377] Logged in from keytab as oryx/127.19.74.194@KRBTEST.COM (short username oryx)
I20260806 22:42:34.076259 26264 webserver.cc:460] Webserver started at http://127.19.74.194:39363/ using document root <none> and password file <none>
I20260806 22:42:34.076440 26264 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.076479 26264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.076566 26264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:34.077512 26264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "0b18d85f26ca4753b60c11a7a974a693"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "c48804c411ea261a179e55d80ae7a41a"
server_key_iv: "5c4fd52f0332e65bf8c2c76f5be2f5b1"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.077812 26264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "0b18d85f26ca4753b60c11a7a974a693"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "c48804c411ea261a179e55d80ae7a41a"
server_key_iv: "5c4fd52f0332e65bf8c2c76f5be2f5b1"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.079121 26264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:34.079799 26279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.079967 26264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:34.080040 26264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "0b18d85f26ca4753b60c11a7a974a693"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "c48804c411ea261a179e55d80ae7a41a"
server_key_iv: "5c4fd52f0332e65bf8c2c76f5be2f5b1"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.080092 26264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.090406 26264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:34.092619 26264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:34.092727 26264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:34.092898 26264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:34.093132 26264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:34.093161 26264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.093182 26264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:34.093195 26264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.099290 26264 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:34653
I20260806 22:42:34.099339 26397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:34653 every 8 connection(s)
I20260806 22:42:34.099746 26264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:34.107476 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 26264
I20260806 22:42:34.107594 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:34.107841 19755 external_mini_cluster.cc:1428] Setting key eea22eee3bc00c303db47ff220cd8e30
I20260806 22:42:34.109853 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.100881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:46377)
0806 22:42:34.101097 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.101170 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.101801 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.102017 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.102020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.102582 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.102863 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.102868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.103508 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.103508 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.104286 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.104288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.104615 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.104616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.104715 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.105252 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.105261 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.106600 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.108495 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.108502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.108505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.108761 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.108974 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.108974 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.108974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.109073 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.109360 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.109365 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.109475 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.109558 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.109687 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":108,"threads_started":1}
I20260806 22:42:34.109851 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.100918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:46377 (local address 127.19.74.254:42469)
0806 22:42:34.100996 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.100999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.101253 (+ 254us) server_negotiation.cc:407] Connection header received
0806 22:42:34.101899 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.101901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.101912 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.101966 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.102938 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.103428 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.104359 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.104538 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.104589 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.106732 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.106733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.106734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.106740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.108385 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.108839 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.108840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.108841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.108906 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.109138 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.109138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.109139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.109294 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.109367 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.109602 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.109715 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:42:34.110321 26398 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42469
I20260806 22:42:34.110451 26398 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:34.110711 26398 heartbeater.cc:511] Master 127.19.74.254:42469 requested a full tablet report, sending...
I20260806 22:42:34.111124 26062 ts_manager.cc:194] Registered new tserver with Master: 0b18d85f26ca4753b60c11a7a974a693 (127.19.74.194:34653)
I20260806 22:42:34.111662 26062 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:46377
WARNING: no policy specified for oryx/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab.
Entry for principal oryx/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab.
I20260806 22:42:34.135245 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42469
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab
--principal=oryx/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:34.217676 26403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:34.217869 26403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:34.217892 26403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:34.219075 26403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:34.219117 26403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:34.219142 26403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:34.220583 26403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab
--principal=oryx/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:34.220849 26403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:34.221081 26403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:34.226189 26413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.226199 26410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.226199 26411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.226397 26403 server_base.cc:1034] running on GCE node
I20260806 22:42:34.226652 26403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:34.226902 26403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:34.228042 26403 hybrid_clock.cc:648] HybridClock initialized: now 1786056154228030 us; error 29 us; skew 500 ppm
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:34.230561 26403 init.cc:377] Logged in from keytab as oryx/127.19.74.195@KRBTEST.COM (short username oryx)
I20260806 22:42:34.231065 26403 webserver.cc:460] Webserver started at http://127.19.74.195:41261/ using document root <none> and password file <none>
I20260806 22:42:34.231323 26403 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.231367 26403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.231520 26403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:34.232543 26403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "88a22fda19bd44f68b6c7b93bd2b74c4"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "5e5fccfe7ded21c6baaeccbc062d1a1d"
server_key_iv: "b1aad660d49d19e244cbafa4817089bc"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.232896 26403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "88a22fda19bd44f68b6c7b93bd2b74c4"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "5e5fccfe7ded21c6baaeccbc062d1a1d"
server_key_iv: "b1aad660d49d19e244cbafa4817089bc"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.234213 26403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:42:34.234988 26420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.235193 26403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.235286 26403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "88a22fda19bd44f68b6c7b93bd2b74c4"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "5e5fccfe7ded21c6baaeccbc062d1a1d"
server_key_iv: "b1aad660d49d19e244cbafa4817089bc"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.235356 26403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.254096 26403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:34.256239 26403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:34.256352 26403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:34.256527 26403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:34.256891 26403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:34.256954 26403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.257019 26403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:34.257048 26403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.262730 26403 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:43029
I20260806 22:42:34.262825 26535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:43029 every 8 connection(s)
I20260806 22:42:34.263132 26403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:34.270934 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 26403
I20260806 22:42:34.271028 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:34.271270 19755 external_mini_cluster.cc:1428] Setting key 7475e6d457c70bec9084e6962c073037
I20260806 22:42:34.272610 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.263919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:33703 (local address 127.19.74.254:42469)
0806 22:42:34.263972 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.263975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.264219 (+ 244us) server_negotiation.cc:407] Connection header received
0806 22:42:34.264707 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.264708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.264718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.264773 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.265661 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.266151 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.267166 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.267347 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.267400 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.269553 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.269554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.269554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.269561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.271079 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.271543 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.271545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.271545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.271607 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.271956 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.271957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.271958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.272115 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.272218 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.272471 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.272540 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:34.272814 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.263909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:33703)
0806 22:42:34.264098 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.264135 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.264623 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.264827 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.264830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.265394 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.265586 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.265590 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.266229 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.266230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.267086 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.267095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.267420 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.267420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.267521 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.267985 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.267994 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.269465 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.271193 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.271199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.271201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.271464 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.271688 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.271689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.271689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.271829 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.272221 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.272227 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.272349 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.272438 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.272560 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260806 22:42:34.273268 26536 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42469
I20260806 22:42:34.273376 26536 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:34.273556 26536 heartbeater.cc:511] Master 127.19.74.254:42469 requested a full tablet report, sending...
I20260806 22:42:34.273934 26062 ts_manager.cc:194] Registered new tserver with Master: 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029)
I20260806 22:42:34.274310 26062 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:33703
I20260806 22:42:34.283123 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.74.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.253.keytab.
Entry for principal oryx/127.19.74.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.253.keytab.
I20260806 22:42:34.308943 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.253:34857
--webserver_interface=127.19.74.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.253.keytab
--principal=oryx/127.19.74.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:42469,127.19.74.253:34857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:34.393414 26542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:34.393630 26542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:34.393653 26542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:34.394759 26542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:34.394797 26542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:34.394814 26542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:34.394827 26542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:34.394841 26542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:34.396265 26542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:42469,127.19.74.253:34857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.253.keytab
--principal=oryx/127.19.74.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.253:34857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.74.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
W20260806 22:42:34.396378 26542 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.74.254:42469,127.19.74.253:34857'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 22:42:34.396487 26542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:34.396673 26542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:34.401657 26549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.401777 26550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.401706 26552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.401657 26542 server_base.cc:1034] running on GCE node
I20260806 22:42:34.402104 26542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:34.402431 26542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:34.403568 26542 hybrid_clock.cc:648] HybridClock initialized: now 1786056154403543 us; error 38 us; skew 500 ppm
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:34.410748 26542 init.cc:377] Logged in from keytab as oryx/127.19.74.253@KRBTEST.COM (short username oryx)
I20260806 22:42:34.411226 26542 webserver.cc:460] Webserver started at http://127.19.74.253:42787/ using document root <none> and password file <none>
I20260806 22:42:34.411417 26542 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.411459 26542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.411549 26542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:34.412561 26542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data/instance:
uuid: "ff7bf2742f8443c58614336a2c3ea399"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "9fd7642d505e8ca01d0673b41149f76d"
server_key_iv: "37f5bbc585cb121c64ff68d1daa0c262"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.412864 26542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal/instance:
uuid: "ff7bf2742f8443c58614336a2c3ea399"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "9fd7642d505e8ca01d0673b41149f76d"
server_key_iv: "37f5bbc585cb121c64ff68d1daa0c262"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.414129 26542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:34.414824 26559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.415007 26542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:34.415086 26542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
uuid: "ff7bf2742f8443c58614336a2c3ea399"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "9fd7642d505e8ca01d0673b41149f76d"
server_key_iv: "37f5bbc585cb121c64ff68d1daa0c262"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.415146 26542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.422259 26542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:34.424127 26542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:34.424237 26542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:34.428211 26542 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.253:34857
I20260806 22:42:34.428242 26612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.253:34857 every 8 connection(s)
I20260806 22:42:34.428614 26542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
I20260806 22:42:34.429597 26613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:34.431120 26613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:34.434332 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 26542
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.253@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.74.253@KRBTEST.COM
I20260806 22:42:34.442162 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.431865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58562 (local address 127.19.74.254:42469)
0806 22:42:34.431976 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.431978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.432260 (+ 282us) server_negotiation.cc:407] Connection header received
0806 22:42:34.432970 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.432972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.432996 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.433063 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.433639 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.434195 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.435744 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.435939 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.436047 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.438695 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.438698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.438698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.438712 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.440530 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.440959 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.440960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.440961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.441028 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.441357 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.441358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.441358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.441567 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.441639 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.441986 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.442075 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:42:34.442451 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.431831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:58562)
0806 22:42:34.432116 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.432155 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.432841 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.433133 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.433136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.433376 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.433559 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.433562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.434330 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.434331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.435649 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.435653 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.436075 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.436076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.436198 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.436731 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.436741 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.438529 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.440638 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.440651 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.440653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.440877 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.441104 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.441105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.441105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.441279 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.441646 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.441654 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.441826 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.441956 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.442197 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
I20260806 22:42:34.443135 26613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:34.444451 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.434929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39700 (local address 127.19.74.253:34857)
0806 22:42:34.435092 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.435101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.435118 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:42:34.435179 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.435181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.435204 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.435274 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.435998 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.436723 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.437872 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.438350 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.438428 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.441019 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.441023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.441023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.441029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.443013 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.443462 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.443465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.443466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.443556 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.443781 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.443781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.443782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.443931 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.444123 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.444169 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.444343 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.253@KRBTEST.COM for oryx/127.19.74.253@KRBTEST.COM
I20260806 22:42:34.450696 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.443400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.0.0.1:39706)
0806 22:42:34.443430 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.443440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.443478 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.443737 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.443738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.443868 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.443910 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.443912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.444859 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.444860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.445426 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.445428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.445819 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.445820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.445887 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.446219 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.446222 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.447359 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.449592 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.449593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.449594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.449786 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.450008 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.450009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.450009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.450079 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.450349 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.450349 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.450387 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.450459 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.450559 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 22:42:34.450795 26618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.443435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39706 (local address 127.19.74.253:34857)
0806 22:42:34.443605 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.443607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.443615 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:34.443636 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.443636 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.443647 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.443721 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.443978 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.444763 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.445526 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.445734 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.445820 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.447461 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.447462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.447462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.447469 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.449419 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.449872 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.449873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.449873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.449936 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.450160 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.450161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.450161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.450277 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.450359 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.450495 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.450640 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260806 22:42:34.452062 26613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Bootstrap starting.
I20260806 22:42:34.452795 26613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:34.453095 26613 log.cc:824] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:34.453830 26613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: No bootstrap required, opened a new log
I20260806 22:42:34.455390 26613 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } }
I20260806 22:42:34.455529 26613 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:34.455597 26613 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7bf2742f8443c58614336a2c3ea399, State: Initialized, Role: FOLLOWER
I20260806 22:42:34.455727 26613 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } }
I20260806 22:42:34.455997 26620 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } } }
I20260806 22:42:34.456051 26613 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:34.456112 26620 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:42:34.457777 26631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:34.457850 26631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:34.459429 26542 master_runner.cc:428] Detected that this master ff7bf2742f8443c58614336a2c3ea399 is joining an existing cluster
I20260806 22:42:34.459486 26542 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.74.253:34857
I20260806 22:42:34.467933 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.460669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.19.74.254:42469)
0806 22:42:34.460753 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.460755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.460885 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:42:34.460910 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.460911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.460922 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.461005 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.462243 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.462951 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.463794 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.464073 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.464152 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.465080 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.465081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.465081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.465089 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.466687 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.467077 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.467078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.467079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.467146 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.467401 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.467402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.467402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.467536 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.467612 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.467766 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.467836 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:42:34.467978 26618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.460711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39708 (local address 127.19.74.253:34857)
0806 22:42:34.460783 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.460785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.460857 (+ 72us) server_negotiation.cc:407] Connection header received
0806 22:42:34.460902 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.460902 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.460910 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.460940 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.461456 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.462123 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.462972 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.463264 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.463332 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.464713 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.464714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.464714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.464721 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.466527 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.467094 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.467095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.467095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.467151 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.467376 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.467377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.467377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.467507 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.467554 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.467775 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.467892 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":58}
I20260806 22:42:34.468209 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.460567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.0.0.1:39708)
0806 22:42:34.460775 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.460786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.460848 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.461003 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.461004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.461285 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.461371 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.461373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.462204 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.462205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.462879 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.462881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.463368 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.463369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.463435 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.464244 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.464248 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.464623 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.466662 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.466663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.466663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.466997 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.467226 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.467227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.467227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.467305 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.467600 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.467601 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.467647 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.467955 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.468098 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260806 22:42:34.468469 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.460569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:58570)
0806 22:42:34.460775 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.460786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.460848 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.461005 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.461006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.461282 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.461371 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.461373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.463041 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.463041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.463706 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.463709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.464151 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.464152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.464211 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.464577 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.464581 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.464999 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.466759 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.466759 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.466760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.467003 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.467228 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.467229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.467230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.467340 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.467607 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.467607 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.467650 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.467737 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.468370 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.254@KRBTEST.COM for oryx/127.19.74.253@KRBTEST.COM
I20260806 22:42:34.478317 26618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.471075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39722 (local address 127.19.74.253:34857)
0806 22:42:34.471112 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.471114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.471255 (+ 141us) server_negotiation.cc:407] Connection header received
0806 22:42:34.471355 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.471355 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.471362 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.471402 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.471920 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.472338 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.473401 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.473562 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.473616 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.475682 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.475683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.475684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.475690 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.477108 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.477484 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.477485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.477485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.477538 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.477762 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.477763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.477764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.477882 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.477951 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.478152 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.478227 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:34.478338 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.470986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.0.0.1:39722)
0806 22:42:34.471128 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.471180 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.471276 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.471423 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.471425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.471782 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.471839 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.471842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.472421 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.472422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.473316 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.473319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.473637 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.473638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.473720 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.474120 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.474125 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.475594 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.477199 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.477206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.477208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.477407 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.477629 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.477630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.477631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.477691 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.477956 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.477964 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.478013 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.478122 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.478230 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260806 22:42:34.478876 26062 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.74.253:34857: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399"
member_type: NON_VOTER
last_known_addr {
host: "127.19.74.253"
port: 34857
}
attrs {
promote: true
}
}
dest_uuid: "9854708559ab44a9a626401c6b3a2499"
cas_config_opid_index: -1
I20260806 22:42:34.479187 26062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } }
I20260806 22:42:34.479648 26642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ff7bf2742f8443c58614336a2c3ea399 (127.19.74.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } }
I20260806 22:42:34.480018 26587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:34.480755 26642 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.74.253:34857
I20260806 22:42:34.480845 26642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } }
I20260806 22:42:34.480917 26642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:34.480965 26587 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Refusing update from remote peer 9854708559ab44a9a626401c6b3a2499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 22:42:34.481117 26542 master.cc:537] Master@127.19.74.253:34857 shutting down...
I20260806 22:42:34.481356 26641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:42:34.482349 26049 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 22:42:34.483873 26542 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:34.483961 26542 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:34.483997 26542 tablet_replica.cc:330] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: stopping tablet replica
I20260806 22:42:34.486961 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.486859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.0.0.1:39724)
0806 22:42:34.486905 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.486938 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:34857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:42:34.487056 26049 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 -> Peer ff7bf2742f8443c58614336a2c3ea399 (127.19.74.253:34857): Couldn't send request to peer ff7bf2742f8443c58614336a2c3ea399. Status: Network error: Client connection negotiation failed: client connection to 127.19.74.253:34857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:42:34.496734 26542 master.cc:552] Master@127.19.74.253:34857 shutdown complete.
I20260806 22:42:34.496790 26542 master_runner.cc:305] Clearing existing system tablet
I20260806 22:42:34.496924 26542 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.496973 26542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.497710 26542 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260806 22:42:34.498253 26648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.498385 26542 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.498451 26542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.498486 26542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
uuid: "ff7bf2742f8443c58614336a2c3ea399"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "9fd7642d505e8ca01d0673b41149f76d"
server_key_iv: "37f5bbc585cb121c64ff68d1daa0c262"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.499183 26542 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:34.500797 26542 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:42:34.500856 26542 log.cc:1196] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 22:42:34.501051 26542 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Deleting consensus metadata
I20260806 22:42:34.501190 26542 master_runner.cc:315] Copying system tablet from 127.19.74.254:42469
I20260806 22:42:34.501796 26542 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: tablet copy: Beginning tablet copy session from remote peer at address 127.19.74.254:42469
I20260806 22:42:34.508208 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.502053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:58582)
0806 22:42:34.502228 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.502237 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.502289 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.502432 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.502433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.502587 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.502747 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.502749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.503705 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.503705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.504266 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.504267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.504731 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.504732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.504800 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.505281 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.505286 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.505584 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.507260 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.507261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.507261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.507432 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.507639 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.507639 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.507639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.507691 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.507980 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.507981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.508014 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.508059 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.508132 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260806 22:42:34.508253 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.502152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.19.74.254:42469)
0806 22:42:34.502212 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.502215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.502278 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:42:34.502354 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.502355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.502366 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.502423 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.502823 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.503631 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.504346 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.504658 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.504728 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.505672 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.505674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.505674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.505680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.507186 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.507502 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.507503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.507504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.507566 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.507769 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.507770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.507771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.507905 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.507986 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.508092 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.508168 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:42:34.508826 26084 tablet_copy_service.cc:140] P 9854708559ab44a9a626401c6b3a2499: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ff7bf2742f8443c58614336a2c3ea399 ({username='oryx', principal='oryx/127.19.74.253@KRBTEST.COM'} at 127.0.0.1:58582)
I20260806 22:42:34.508942 26084 tablet_copy_service.cc:161] P 9854708559ab44a9a626401c6b3a2499: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ff7bf2742f8443c58614336a2c3ea399 at {username='oryx', principal='oryx/127.19.74.253@KRBTEST.COM'} at 127.0.0.1:58582: session id = ff7bf2742f8443c58614336a2c3ea399-00000000000000000000000000000000
I20260806 22:42:34.509541 26084 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:42:34.510126 26542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:42:34.511099 26542 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: tablet copy: Starting download of 0 data blocks...
I20260806 22:42:34.511263 26542 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: tablet copy: Starting download of 1 WAL segments...
I20260806 22:42:34.512113 26542 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:42:34.512995 26084 tablet_copy_service.cc:342] P 9854708559ab44a9a626401c6b3a2499: Request end of tablet copy session ff7bf2742f8443c58614336a2c3ea399-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.74.253@KRBTEST.COM'} at 127.0.0.1:58582
I20260806 22:42:34.513055 26084 tablet_copy_service.cc:433] P 9854708559ab44a9a626401c6b3a2499: ending tablet copy session ff7bf2742f8443c58614336a2c3ea399-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ff7bf2742f8443c58614336a2c3ea399
I20260806 22:42:34.515363 26542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:34.516415 26655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.516515 26656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.519464 26658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.519575 26542 server_base.cc:1034] running on GCE node
I20260806 22:42:34.519661 26542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:34.519833 26542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:34.520951 26542 hybrid_clock.cc:648] HybridClock initialized: now 1786056154520925 us; error 43 us; skew 500 ppm
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:34.522210 26542 init.cc:377] Logged in from keytab as oryx/127.19.74.253@KRBTEST.COM (short username oryx)
I20260806 22:42:34.522497 26542 webserver.cc:460] Webserver started at http://127.19.74.253:36763/ using document root <none> and password file <none>
I20260806 22:42:34.522662 26542 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.522729 26542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.523437 26542 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:34.523983 26665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.524137 26542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:34.524184 26542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
uuid: "ff7bf2742f8443c58614336a2c3ea399"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "9fd7642d505e8ca01d0673b41149f76d"
server_key_iv: "37f5bbc585cb121c64ff68d1daa0c262"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.524240 26542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.534061 26542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:34.535691 26542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:34.538739 26542 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.253:34857
I20260806 22:42:34.538937 26718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.253:34857 every 8 connection(s)
I20260806 22:42:34.539098 26542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
I20260806 22:42:34.540091 26719 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:42:34.540642 26719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Bootstrap starting.
I20260806 22:42:34.543583 26719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:42:34.543839 26719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Bootstrap complete.
I20260806 22:42:34.544072 26719 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } }
I20260806 22:42:34.544164 26719 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff7bf2742f8443c58614336a2c3ea399, State: Initialized, Role: LEARNER
I20260806 22:42:34.544222 26719 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } }
I20260806 22:42:34.544400 26719 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:34.544409 26723 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } }
I20260806 22:42:34.544575 26723 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:42:34.546109 26734 catalog_manager.cc:1259] Loaded cluster ID: dc5f99a5646847c789e5f4e81ec45ea9
I20260806 22:42:34.546384 26734 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: loading cluster ID for follower catalog manager: success
I20260806 22:42:34.547360 26734 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: acquiring CA information for follower catalog manager: success
I20260806 22:42:34.547633 26734 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:42:34.557449 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.546034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.19.74.253:34857)
0806 22:42:34.550723 (+ 4689us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.550725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.550734 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:34.550754 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.550754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.550763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.550827 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.551574 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.552172 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.553224 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.553407 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.553456 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.554895 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.554896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.554896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.554902 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.556321 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.556723 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.556724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.556724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.556775 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.556987 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.556988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.556988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.557078 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.557157 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.557318 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.557380 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4664,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for oryx/127.19.74.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.252.keytab.
Entry for principal oryx/127.19.74.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.252.keytab.
I20260806 22:42:34.581421 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.252:43901
--webserver_interface=127.19.74.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.252.keytab
--principal=oryx/127.19.74.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:42469,127.19.74.253:34857,127.19.74.252:43901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:34.664446 26738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:34.664670 26738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:34.664708 26738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:34.665822 26738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:34.665889 26738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:34.665912 26738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:34.665940 26738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:34.665970 26738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:34.667564 26738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:42469,127.19.74.253:34857,127.19.74.252:43901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.252.keytab
--principal=oryx/127.19.74.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.252:43901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.74.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:34.667842 26738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:34.668111 26738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:34.673171 26744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.673147 26746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.673372 26738 server_base.cc:1034] running on GCE node
W20260806 22:42:34.673147 26743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.673676 26738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:34.674002 26738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:34.675168 26738 hybrid_clock.cc:648] HybridClock initialized: now 1786056154675152 us; error 26 us; skew 500 ppm
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:34.677570 26738 init.cc:377] Logged in from keytab as oryx/127.19.74.252@KRBTEST.COM (short username oryx)
I20260806 22:42:34.677986 26738 webserver.cc:460] Webserver started at http://127.19.74.252:40543/ using document root <none> and password file <none>
I20260806 22:42:34.678239 26738 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.678309 26738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.678459 26738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:34.679538 26738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data/instance:
uuid: "e36883e05a7e4f5ebd44e7415ac5c536"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "e0d46ea4ef4899f6f1ffd0dc80ccc69c"
server_key_iv: "fed17393c1575c1ba0edd1ca2e6f5e21"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.679904 26738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal/instance:
uuid: "e36883e05a7e4f5ebd44e7415ac5c536"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "e0d46ea4ef4899f6f1ffd0dc80ccc69c"
server_key_iv: "fed17393c1575c1ba0edd1ca2e6f5e21"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.681245 26738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:34.681957 26753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.682147 26738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:34.682256 26738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
uuid: "e36883e05a7e4f5ebd44e7415ac5c536"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "e0d46ea4ef4899f6f1ffd0dc80ccc69c"
server_key_iv: "fed17393c1575c1ba0edd1ca2e6f5e21"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.682328 26738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.695889 26738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:34.697669 26738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:34.697834 26738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:34.701860 26738 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.252:43901
I20260806 22:42:34.701910 26808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.252:43901 every 8 connection(s)
I20260806 22:42:34.702262 26738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
I20260806 22:42:34.703310 26809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:34.704826 26809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:34.706720 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 26738
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.252@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.74.252@KRBTEST.COM
I20260806 22:42:34.714509 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.705669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58598 (local address 127.19.74.254:42469)
0806 22:42:34.705741 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.705743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.705813 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:42:34.706301 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.706302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.706311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.706365 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.707291 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.707840 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.708918 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.709210 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.709299 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.711579 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.711581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.711581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.711588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.713097 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.713547 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.713549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.713549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.713613 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.713868 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.713869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.713869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.714000 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.714076 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.714341 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.714423 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:42:34.714882 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.705503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:58598)
0806 22:42:34.705701 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.705748 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.706217 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.706442 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.706445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.706862 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.707154 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.707161 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.707950 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.707952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.708831 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.708834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.709328 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.709329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.709456 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.709985 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.709995 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.711478 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.713181 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.713188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.713190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.713457 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.713687 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.713688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.713688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.713796 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.714067 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.714073 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.714209 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.714315 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.714453 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
I20260806 22:42:34.715384 26809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:34.715765 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.707250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.19.74.252:43901)
0806 22:42:34.707421 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.707431 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.707446 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:42:34.707464 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.707465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.707483 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.707567 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.708248 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.709026 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.709850 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.710166 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.710221 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.712700 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.712707 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.712708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.712714 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.714228 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.714586 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.714589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.714589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.714667 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.714875 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.714876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.714876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.715078 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.715556 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.715606 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.715682 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.252@KRBTEST.COM for oryx/127.19.74.253@KRBTEST.COM
I20260806 22:42:34.722071 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.715635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.0.0.1:39742)
0806 22:42:34.715748 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.715757 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.715901 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.716072 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.716073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.716234 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.716283 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.716285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.716929 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.716930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.717388 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.717389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.717767 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.717768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.717815 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.718106 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.718110 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.719251 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.720970 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.720971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.720971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.721146 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.721370 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.721371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.721372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.721438 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.721722 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.721723 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.721768 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.721870 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.721964 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 22:42:34.722106 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.715703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39742 (local address 127.19.74.253:34857)
0806 22:42:34.715745 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.715747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.715791 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:42:34.715977 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.715978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.715994 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.716054 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.716367 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.716865 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.717479 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.717691 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.717763 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.719361 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.719362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.719363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.719377 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.720862 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.721235 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.721237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.721237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.721298 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.721527 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.721527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.721528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.721650 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.721730 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.721900 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.722011 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:34.722596 26809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.252@KRBTEST.COM for oryx/127.19.74.252@KRBTEST.COM
I20260806 22:42:34.728907 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.722928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.0.0.1:43742)
0806 22:42:34.723006 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.723014 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.723052 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.723159 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.723160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.723293 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.723315 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.723316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.723866 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.723867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.724401 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.724402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.724787 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.724787 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.724823 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.725466 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.725470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.726545 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.728177 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.728178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.728178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.728317 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.728491 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.728492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.728492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.728539 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.728710 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.728711 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.728741 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.728790 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.728855 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 22:42:34.728968 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.722954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.19.74.252:43901)
0806 22:42:34.722999 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.723001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.723044 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:42:34.723095 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.723095 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.723104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.723156 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.723359 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.723804 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.724506 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.724719 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.724779 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.726662 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.726663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.726664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.726678 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.728107 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.728391 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.728393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.728393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.728437 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.728576 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.728577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.728577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.728660 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.728708 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.728806 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.728898 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":18}
I20260806 22:42:34.730393 26809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Bootstrap starting.
I20260806 22:42:34.731249 26809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:34.731539 26809 log.cc:824] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:34.732223 26809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: No bootstrap required, opened a new log
I20260806 22:42:34.733634 26809 raft_consensus.cc:348] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } }
I20260806 22:42:34.733771 26809 raft_consensus.cc:374] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:34.733821 26809 raft_consensus.cc:729] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36883e05a7e4f5ebd44e7415ac5c536, State: Initialized, Role: FOLLOWER
I20260806 22:42:34.733953 26809 consensus_queue.cc:260] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } }
I20260806 22:42:34.734253 26809 sys_catalog.cc:564] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:34.734222 26814 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } } }
I20260806 22:42:34.734341 26814 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:42:34.735927 26825 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:34.735986 26825 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:34.738260 26738 master_runner.cc:428] Detected that this master e36883e05a7e4f5ebd44e7415ac5c536 is joining an existing cluster
I20260806 22:42:34.738330 26738 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.74.252:43901
I20260806 22:42:34.766147 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.739406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.19.74.252:43901)
0806 22:42:34.739438 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.739439 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.739736 (+ 297us) server_negotiation.cc:407] Connection header received
0806 22:42:34.739756 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.739757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.739766 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.739851 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.740290 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.740965 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.741971 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.742227 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.742264 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.743401 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.743402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.743403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.743410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.764842 (+ 21432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.765221 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.765222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.765223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.765321 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.765529 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.765530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.765531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.765685 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.765735 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.765954 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.766037 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:34.766364 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.739371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39752 (local address 127.19.74.253:34857)
0806 22:42:34.739420 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.739422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.740395 (+ 973us) server_negotiation.cc:407] Connection header received
0806 22:42:34.740478 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.740478 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.740488 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.740515 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.740886 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.741563 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.742565 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.742912 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.742948 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.763475 (+ 20527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.763477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.763478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.763486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.765118 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.765547 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.765548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.765549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.765616 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.765817 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.765818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.765818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.765941 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.765983 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.766187 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.766269 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:42:34.766943 26831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.739475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.0.0.1:39752)
0806 22:42:34.740344 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.740353 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.740417 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.740570 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.740570 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.740751 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.740804 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.740805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.741663 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.741664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.742472 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.742474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.743010 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.743010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.743055 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.743475 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.743478 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.763325 (+ 19847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.765216 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.765217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.765217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.765461 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.765689 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.765690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.765691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.765749 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.766031 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.766031 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.766072 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.766481 (+ 409us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.766625 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"thread_start_us":99,"threads_started":1}
I20260806 22:42:34.767191 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 26126
I20260806 22:42:34.767441 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.739475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.0.0.1:43756)
0806 22:42:34.739624 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.739635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.739698 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.739878 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.739879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.740130 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.740213 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.740215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.741055 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.741056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.741881 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.741884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.742338 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.742339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.742391 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.742863 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.742868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.743289 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.764925 (+ 21636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.764926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.764926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.765144 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.765389 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.765389 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.765390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.765464 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.765783 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.765784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.765840 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.766784 (+ 944us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.766888 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":14,"thread_start_us":72,"threads_started":1}
I20260806 22:42:34.768384 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.739309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58610 (local address 127.19.74.254:42469)
0806 22:42:34.739348 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.739350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.739707 (+ 357us) server_negotiation.cc:407] Connection header received
0806 22:42:34.739760 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.739761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.739789 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.739843 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.740310 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.741015 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.742008 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.742305 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.742341 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.764753 (+ 22412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.764755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.764756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.764764 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.766512 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.767402 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.767403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.767404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.767474 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.767734 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.767735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.767735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.767885 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.767935 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.768194 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.768286 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:34.768747 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.739414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:58610)
0806 22:42:34.739624 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.739635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.739698 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.739920 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.739921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.740113 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.740213 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.740215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.741096 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.741097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.741922 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.741923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.742414 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.742415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.742474 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.763492 (+ 21018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.763499 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.764638 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.766655 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.766657 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.766657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.767303 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.767573 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.767574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.767574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.767659 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.768006 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.768007 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.768049 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.768517 (+ 468us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.768636 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":5,"thread_start_us":93,"threads_started":1}
I20260806 22:42:34.777369 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:37361
--local_ip_for_outbound_sockets=127.19.74.193
--tserver_master_addrs=127.19.74.254:42469
--webserver_port=39691
--webserver_interface=127.19.74.193
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab
--principal=oryx/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857,127.19.74.252:43901 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056153, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.254@KRBTEST.COM for oryx/127.19.74.252@KRBTEST.COM
I20260806 22:42:34.780769 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.773033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.0.0.1:43772)
0806 22:42:34.773110 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.773122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.773168 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.773327 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.773328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.773510 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.773568 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.773570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.774540 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.774541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.775391 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.775394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.775909 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.775910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.775980 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.776431 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.776436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.777842 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.779632 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.779633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.779633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.779838 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.780092 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.780093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.780093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.780161 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.780445 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.780446 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.780491 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.780572 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.780670 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 22:42:34.781081 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.773142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.19.74.252:43901)
0806 22:42:34.773180 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.773182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.773191 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:34.773240 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.773241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.773253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.773329 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.773665 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.774434 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.775497 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.775819 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.775904 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.777932 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.777933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.777933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.777940 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.779542 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.779943 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.779944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.779944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.780019 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.780232 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.780233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.780233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.780373 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.780441 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.780612 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.780696 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:42:34.781562 26062 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.74.252:43901: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536"
member_type: NON_VOTER
last_known_addr {
host: "127.19.74.252"
port: 43901
}
attrs {
promote: true
}
}
dest_uuid: "9854708559ab44a9a626401c6b3a2499"
cas_config_opid_index: 5
I20260806 22:42:34.781855 26062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } }
I20260806 22:42:34.782239 26642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.782786 26781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:34.783473 26642 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.74.252:43901
I20260806 22:42:34.783561 26642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.783648 26642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:34.783700 26781 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Refusing update from remote peer 9854708559ab44a9a626401c6b3a2499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:42:34.784055 26738 master.cc:537] Master@127.19.74.252:43901 shutting down...
I20260806 22:42:34.785578 26641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:42:34.786757 26047 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 -> Peer e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252:43901): Couldn't send request to peer e36883e05a7e4f5ebd44e7415ac5c536. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:42:34.788090 26738 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:34.788187 26738 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:34.788232 26738 tablet_replica.cc:330] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: stopping tablet replica
I20260806 22:42:34.791116 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.782378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.0.0.1:39764)
0806 22:42:34.787393 (+ 5015us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.787416 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.787463 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.787748 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.787749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.787751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:34.787790 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.787792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.788759 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.788760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.789895 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.789898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.790877 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.790878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.790944 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.790966 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.791018 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.791051 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4999}
I20260806 22:42:34.791420 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.782481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.19.74.253:34857)
0806 22:42:34.787360 (+ 4879us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.787363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.787512 (+ 149us) server_negotiation.cc:407] Connection header received
0806 22:42:34.787547 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.787548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.787569 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.787621 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:34.787868 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.788586 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.789989 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.790564 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.790649 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.790720 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.791310 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.791348 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4853}
I20260806 22:42:34.791868 26693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 LEARNER]: Refusing update from remote peer 9854708559ab44a9a626401c6b3a2499: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:42:34.792145 26641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 22:42:34.792727 26693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.793996 26641 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: attempting to promote NON_VOTER ff7bf2742f8443c58614336a2c3ea399 to VOTER
I20260806 22:42:34.794056 26723 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9854708559ab44a9a626401c6b3a2499. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.794134 26723 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:42:34.794167 26723 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: NON_VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.794204 26723 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:42:34.794190 26641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } }
I20260806 22:42:34.794657 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.794547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.0.0.1:43788)
0806 22:42:34.794583 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.794632 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 22:42:34.794754 26693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 LEARNER]: Refusing update from remote peer 9854708559ab44a9a626401c6b3a2499: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:42:34.794899 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.794840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.0.0.1:43802)
0806 22:42:34.794866 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.794883 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 22:42:34.794965 26047 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 -> Peer e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252:43901): Couldn't send request to peer e36883e05a7e4f5ebd44e7415ac5c536. Status: Network error: Client connection negotiation failed: client connection to 127.19.74.252:43901: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:42:34.795078 26641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 22:42:34.795771 26641 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ff7bf2742f8443c58614336a2c3ea399 (127.19.74.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.796761 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.796833 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:34.796880 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.796918 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:34.797252 26693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ff7bf2742f8443c58614336a2c3ea399 (127.19.74.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.798295 26836 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.798415 26836 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:34.801370 26738 master.cc:552] Master@127.19.74.252:43901 shutdown complete.
I20260806 22:42:34.801421 26738 master_runner.cc:305] Clearing existing system tablet
I20260806 22:42:34.801564 26738 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.801611 26738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.802307 26738 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:34.802861 26841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.802973 26738 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.803018 26738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.803040 26738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
uuid: "e36883e05a7e4f5ebd44e7415ac5c536"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "e0d46ea4ef4899f6f1ffd0dc80ccc69c"
server_key_iv: "fed17393c1575c1ba0edd1ca2e6f5e21"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.803707 26738 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:34.804991 26738 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:42:34.805060 26738 log.cc:1196] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 22:42:34.805241 26738 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Deleting consensus metadata
I20260806 22:42:34.805387 26738 master_runner.cc:315] Copying system tablet from 127.19.74.254:42469
I20260806 22:42:34.805989 26738 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: tablet copy: Beginning tablet copy session from remote peer at address 127.19.74.254:42469
I20260806 22:42:34.812783 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.806246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:58616)
0806 22:42:34.806340 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.806349 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.806400 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.806565 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.806566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.806780 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.806824 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.806826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.807745 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.807746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.808509 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.808512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.808965 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.808966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.809027 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.809500 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.809506 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.809946 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.811665 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.811666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.811666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.811892 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.812102 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.812103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.812103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.812168 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.812469 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.812470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.812514 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.812595 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.812692 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
I20260806 22:42:34.812790 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.806287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58616 (local address 127.19.74.254:42469)
0806 22:42:34.806326 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.806328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.806387 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:42:34.806447 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.806448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.806457 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.806560 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.806903 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.807653 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.808608 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.808882 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.808952 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.810038 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.810039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.810040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.810047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.811580 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.811967 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.811968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.811968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.812030 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.812255 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.812255 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.812256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.812401 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.812465 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.812632 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.812711 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:34.813370 26084 tablet_copy_service.cc:140] P 9854708559ab44a9a626401c6b3a2499: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e36883e05a7e4f5ebd44e7415ac5c536 ({username='oryx', principal='oryx/127.19.74.252@KRBTEST.COM'} at 127.0.0.1:58616)
I20260806 22:42:34.813431 26084 tablet_copy_service.cc:161] P 9854708559ab44a9a626401c6b3a2499: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e36883e05a7e4f5ebd44e7415ac5c536 at {username='oryx', principal='oryx/127.19.74.252@KRBTEST.COM'} at 127.0.0.1:58616: session id = e36883e05a7e4f5ebd44e7415ac5c536-00000000000000000000000000000000
I20260806 22:42:34.814016 26084 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:42:34.814530 26738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:42:34.815824 26738 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: tablet copy: Starting download of 0 data blocks...
I20260806 22:42:34.815972 26738 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: tablet copy: Starting download of 1 WAL segments...
I20260806 22:42:34.816684 26738 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:42:34.817562 26084 tablet_copy_service.cc:342] P 9854708559ab44a9a626401c6b3a2499: Request end of tablet copy session e36883e05a7e4f5ebd44e7415ac5c536-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.74.252@KRBTEST.COM'} at 127.0.0.1:58616
I20260806 22:42:34.817618 26084 tablet_copy_service.cc:433] P 9854708559ab44a9a626401c6b3a2499: ending tablet copy session e36883e05a7e4f5ebd44e7415ac5c536-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e36883e05a7e4f5ebd44e7415ac5c536
I20260806 22:42:34.820024 26738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:34.821184 26848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.823500 26849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.823666 26851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.823977 26738 server_base.cc:1034] running on GCE node
I20260806 22:42:34.824117 26738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:34.824254 26738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:34.825403 26738 hybrid_clock.cc:648] HybridClock initialized: now 1786056154825378 us; error 41 us; skew 500 ppm
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:34.827135 26738 init.cc:377] Logged in from keytab as oryx/127.19.74.252@KRBTEST.COM (short username oryx)
I20260806 22:42:34.827461 26738 webserver.cc:460] Webserver started at http://127.19.74.252:36129/ using document root <none> and password file <none>
I20260806 22:42:34.827608 26738 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.827643 26738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.828395 26738 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:34.829010 26858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.829185 26738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.829236 26738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
uuid: "e36883e05a7e4f5ebd44e7415ac5c536"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "e0d46ea4ef4899f6f1ffd0dc80ccc69c"
server_key_iv: "fed17393c1575c1ba0edd1ca2e6f5e21"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.829294 26738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.843673 26738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:34.845664 26738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:34.848796 26738 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.252:43901
I20260806 22:42:34.849267 26738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
I20260806 22:42:34.850751 26912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.252:43901 every 8 connection(s)
I20260806 22:42:34.851471 26913 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:42:34.851949 26913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Bootstrap starting.
I20260806 22:42:34.854840 26913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:42:34.855048 26913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: Bootstrap complete.
I20260806 22:42:34.855274 26913 raft_consensus.cc:348] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } }
I20260806 22:42:34.855373 26913 raft_consensus.cc:729] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e36883e05a7e4f5ebd44e7415ac5c536, State: Initialized, Role: LEARNER
I20260806 22:42:34.855412 26913 consensus_queue.cc:260] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } }
I20260806 22:42:34.855625 26913 sys_catalog.cc:564] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:34.855634 26916 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:34.855855 26916 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:42:34.857092 26927 catalog_manager.cc:1259] Loaded cluster ID: dc5f99a5646847c789e5f4e81ec45ea9
I20260806 22:42:34.857179 26927 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: loading cluster ID for follower catalog manager: success
I20260806 22:42:34.858791 26927 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: acquiring CA information for follower catalog manager: success
I20260806 22:42:34.859169 26927 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 22:42:34.882704 26834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:34.882958 26834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:34.882998 26834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:34.884167 26834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:34.884208 26834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:34.884235 26834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:34.885707 26834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab
--principal=oryx/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:37361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=39691
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857,127.19.74.252:43901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:34.885937 26834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:34.886204 26834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:34.891734 26933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.891808 26935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.891832 26932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.892133 26834 server_base.cc:1034] running on GCE node
I20260806 22:42:34.892378 26834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:34.892619 26834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:34.893810 26834 hybrid_clock.cc:648] HybridClock initialized: now 1786056154893777 us; error 45 us; skew 500 ppm
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:34.896763 26834 init.cc:377] Logged in from keytab as oryx/127.19.74.193@KRBTEST.COM (short username oryx)
I20260806 22:42:34.897246 26834 webserver.cc:460] Webserver started at http://127.19.74.193:39691/ using document root <none> and password file <none>
I20260806 22:42:34.897481 26834 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.897573 26834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.899062 26834 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:34.899713 26942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.899891 26834 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:34.900002 26834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "0df6c1f5bac54740a8d25e6118c6cdac"
format_stamp: "Formatted at 2026-08-06 22:42:33 on dist-test-slave-pgkr"
server_key: "7d9d29cfb5e325780ce54716d0c90be2"
server_key_iv: "d693e3a8ff17be5b86079bc9e52d965f"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.900357 26834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.935231 26834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:34.937089 26834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:34.937263 26834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:34.937507 26834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:34.937871 26834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:34.937944 26834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.937981 26834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:34.938009 26834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.943872 26834 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:37361
I20260806 22:42:34.943919 27060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:37361 every 8 connection(s)
I20260806 22:42:34.944253 26834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.253@KRBTEST.COM
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
Aug 06 22:42:34 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.252@KRBTEST.COM
I20260806 22:42:34.954260 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 26834
I20260806 22:42:34.954423 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 26264
I20260806 22:42:34.957888 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.945089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:59007 (local address 127.19.74.254:42469)
0806 22:42:34.945139 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.945141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.947022 (+ 1881us) server_negotiation.cc:407] Connection header received
0806 22:42:34.947121 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.947122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.947137 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.947215 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.947615 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.948066 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.949324 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.949548 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.949639 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.953002 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.953003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.953004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.953013 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.954761 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.956613 (+ 1852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.956615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.956615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.956690 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.957037 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.957038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.957038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.957191 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.957252 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.957674 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.957770 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:42:34.958439 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.944964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:59007)
0806 22:42:34.946958 (+ 1994us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.946968 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.947050 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.947221 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.947222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.947414 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.947472 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.947474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.948185 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.948186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.949208 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.949211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.949646 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.949647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.949792 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.950506 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.950510 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.952875 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.956111 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.956112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.956112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.956519 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.956778 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.956779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.956780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.956957 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.957267 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.957275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.957475 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.957628 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.957855 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1955,"mutex_wait_us":67,"thread_start_us":89,"threads_started":1}
I20260806 22:42:34.958508 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.945310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:39577 (local address 127.19.74.252:43901)
0806 22:42:34.945460 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.945462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.946830 (+ 1368us) server_negotiation.cc:407] Connection header received
0806 22:42:34.946909 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.946910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.946927 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.947026 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.948177 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.949165 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.950173 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.950530 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.950621 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.953608 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.953609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.953610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.953618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.956911 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.957505 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.957507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.957507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.957580 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.957844 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.957845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.957845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.957994 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.958097 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.958290 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.958378 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260806 22:42:34.958549 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.944964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.19.74.193:39577)
0806 22:42:34.946737 (+ 1773us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.946747 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.946841 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.947039 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.947040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.947214 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.947428 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.947434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.949298 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.949299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.950083 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.950085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.950654 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.950655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.950729 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.951195 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.951200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.953473 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.957031 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.957032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.957032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.957406 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.957691 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.957692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.957692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.957760 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.958094 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.958095 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.958139 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.958260 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.958365 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1737,"mutex_wait_us":30,"thread_start_us":66,"threads_started":1}
I20260806 22:42:34.959435 27062 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42469
I20260806 22:42:34.959542 27062 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:34.959761 27062 heartbeater.cc:511] Master 127.19.74.254:42469 requested a full tablet report, sending...
I20260806 22:42:34.960225 26062 ts_manager.cc:194] Re-registered known tserver with Master: 0df6c1f5bac54740a8d25e6118c6cdac (127.19.74.193:37361)
I20260806 22:42:34.960634 26062 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:59007
I20260806 22:42:34.963100 27061 heartbeater.cc:347] Connected to a master server at 127.19.74.252:43901
I20260806 22:42:34.963161 27061 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:34.963234 27061 heartbeater.cc:511] Master 127.19.74.252:43901 requested a full tablet report, sending...
I20260806 22:42:34.963599 26870 ts_manager.cc:194] Registered new tserver with Master: 0df6c1f5bac54740a8d25e6118c6cdac (127.19.74.193:37361)
I20260806 22:42:34.963726 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:34653
--local_ip_for_outbound_sockets=127.19.74.194
--tserver_master_addrs=127.19.74.254:42469
--webserver_port=39363
--webserver_interface=127.19.74.194
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab
--principal=oryx/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857,127.19.74.252:43901 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:34.964430 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.945251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:38407 (local address 127.19.74.253:34857)
0806 22:42:34.945296 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.945298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.945522 (+ 224us) server_negotiation.cc:407] Connection header received
0806 22:42:34.946649 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.946650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.946663 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.946708 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.947540 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.948053 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.949236 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.949543 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.949648 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.952479 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.952480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.952481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.952489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.954283 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.963479 (+ 9196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.963481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.963481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.963552 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.963793 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.963794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.963794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.963974 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.964026 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.964251 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.964335 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:34.964450 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.944964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.19.74.193:38407)
0806 22:42:34.945374 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:34.945419 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:34.946505 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:34.946779 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:34.946783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:34.947102 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:34.947428 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.947434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.948180 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.948181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.949142 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:34.949149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.949721 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.949722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:34.949796 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.950413 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:34.950421 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:34.952378 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:34.954402 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.954409 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.954411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.963392 (+ 8981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.963637 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:34.963637 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:34.963638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:34.963707 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:34.964078 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:34.964079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:34.964122 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:42:34.964204 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:42:34.964336 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":33,"thread_start_us":87,"threads_started":1}
I20260806 22:42:34.965041 27063 heartbeater.cc:347] Connected to a master server at 127.19.74.253:34857
I20260806 22:42:34.965116 27063 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:34.965199 27063 heartbeater.cc:511] Master 127.19.74.253:34857 requested a full tablet report, sending...
I20260806 22:42:34.965660 26683 ts_manager.cc:194] Registered new tserver with Master: 0df6c1f5bac54740a8d25e6118c6cdac (127.19.74.193:37361)
W20260806 22:42:35.046700 27069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:35.046906 27069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:35.046931 27069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:35.048017 27069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:35.048053 27069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:35.048077 27069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:35.049644 27069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab
--principal=oryx/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:34653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=39363
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857,127.19.74.252:43901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:35.049846 27069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:35.050016 27069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:35.055179 27075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:35.055190 27074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:35.055186 27077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:35.055403 27069 server_base.cc:1034] running on GCE node
I20260806 22:42:35.055691 27069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:35.055981 27069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:35.057157 27069 hybrid_clock.cc:648] HybridClock initialized: now 1786056155057143 us; error 28 us; skew 500 ppm
Aug 06 22:42:35 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:35.059703 27069 init.cc:377] Logged in from keytab as oryx/127.19.74.194@KRBTEST.COM (short username oryx)
I20260806 22:42:35.060127 27069 webserver.cc:460] Webserver started at http://127.19.74.194:39363/ using document root <none> and password file <none>
I20260806 22:42:35.060307 27069 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:35.060359 27069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:35.061621 27069 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:35.062155 27084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:35.062330 27069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:35.062417 27069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "0b18d85f26ca4753b60c11a7a974a693"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "c48804c411ea261a179e55d80ae7a41a"
server_key_iv: "5c4fd52f0332e65bf8c2c76f5be2f5b1"
server_key_version: "encryptionkey@0"
I20260806 22:42:35.062775 27069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:35.071138 27069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:35.072810 27069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:35.072913 27069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:35.073082 27069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:35.073328 27069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:35.073357 27069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:35.073378 27069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:35.073391 27069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:35.078961 27069 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:34653
I20260806 22:42:35.079030 27200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:34653 every 8 connection(s)
I20260806 22:42:35.079370 27069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:42:35 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
Aug 06 22:42:35 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.252@KRBTEST.COM
Aug 06 22:42:35 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.253@KRBTEST.COM
I20260806 22:42:35.089187 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 27069
I20260806 22:42:35.089352 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 26403
I20260806 22:42:35.090341 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.080269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:38725 (local address 127.19.74.254:42469)
0806 22:42:35.080324 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.080326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.080685 (+ 359us) server_negotiation.cc:407] Connection header received
0806 22:42:35.081213 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.081214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.081228 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.081261 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.081969 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.082518 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.083932 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.084236 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.084333 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.086563 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.086564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.086564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.086572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.088574 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.089239 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.089240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.089241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.089302 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.089551 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.089553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.089553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.089706 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.089746 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.090076 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.090196 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:42:35.090517 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.080372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:53277 (local address 127.19.74.252:43901)
0806 22:42:35.080416 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.080418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.080688 (+ 270us) server_negotiation.cc:407] Connection header received
0806 22:42:35.081190 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.081191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.081206 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.081264 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.081962 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.082821 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.083741 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.084031 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.084097 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.087085 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.087086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.087086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.087094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.088623 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.089184 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.089186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.089186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.089251 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.089531 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.089532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.089533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.089677 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.089712 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.090310 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.090402 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:35.091001 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.080141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.19.74.194:53277)
0806 22:42:35.080558 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.080597 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.081087 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.081337 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.081340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.081667 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:35.081856 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.081860 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.082946 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.082947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.083648 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.083650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.084122 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.084123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.084299 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.084793 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:35.084802 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:35.086994 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:35.088704 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.088705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.088705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.089096 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.089332 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.089333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.089333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.089465 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.090168 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:35.090169 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:35.090207 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.090673 (+ 466us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.090793 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":159,"threads_started":1}
I20260806 22:42:35.091008 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.080141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:38725)
0806 22:42:35.080568 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.080596 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.081120 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.081341 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.081341 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.081693 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:35.081856 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.081863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.082661 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.082664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.083835 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.083838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.084333 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.084334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.084402 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.085072 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:35.085077 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:35.086470 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:35.088662 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.088670 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.088672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.089128 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.089362 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.089363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.089364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.089469 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.089788 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:35.089793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:35.089932 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.090447 (+ 515us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.090688 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":8,"thread_start_us":259,"threads_started":1}
I20260806 22:42:35.096385 27202 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42469
I20260806 22:42:35.096490 27202 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:35.096628 27201 heartbeater.cc:347] Connected to a master server at 127.19.74.252:43901
I20260806 22:42:35.096686 27201 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:35.096701 27202 heartbeater.cc:511] Master 127.19.74.254:42469 requested a full tablet report, sending...
I20260806 22:42:35.096768 27201 heartbeater.cc:511] Master 127.19.74.252:43901 requested a full tablet report, sending...
I20260806 22:42:35.097066 26062 ts_manager.cc:194] Re-registered known tserver with Master: 0b18d85f26ca4753b60c11a7a974a693 (127.19.74.194:34653)
I20260806 22:42:35.097162 26870 ts_manager.cc:194] Registered new tserver with Master: 0b18d85f26ca4753b60c11a7a974a693 (127.19.74.194:34653)
I20260806 22:42:35.097582 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.080300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:54897 (local address 127.19.74.253:34857)
0806 22:42:35.080350 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.080352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.080684 (+ 332us) server_negotiation.cc:407] Connection header received
0806 22:42:35.081213 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.081214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.081227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.081261 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.082022 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.082821 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.083960 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.084237 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.084318 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.088159 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.088160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.088160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.088167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.095912 (+ 7745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.096375 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.096376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.096377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.096439 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.096813 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.096814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.096814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.096963 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.097009 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.097387 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.097472 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:42:35.097644 26062 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:38725
I20260806 22:42:35.097918 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.080142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.19.74.194:54897)
0806 22:42:35.080625 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.080637 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.081107 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.081739 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.081740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.081886 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:35.081945 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.081946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.082929 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.082930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.083841 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.083842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.084373 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.084375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.084438 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.084901 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:35.084905 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:35.087622 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:35.096008 (+ 8386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.096009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.096009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.096285 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.096582 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.096582 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.096583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.096650 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.097100 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:35.097101 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:35.097199 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.097687 (+ 488us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.097819 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":99,"thread_start_us":302,"threads_started":1}
I20260806 22:42:35.098703 27203 heartbeater.cc:347] Connected to a master server at 127.19.74.253:34857
I20260806 22:42:35.098765 27203 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:35.098840 27203 heartbeater.cc:511] Master 127.19.74.253:34857 requested a full tablet report, sending...
I20260806 22:42:35.099155 26683 ts_manager.cc:194] Registered new tserver with Master: 0b18d85f26ca4753b60c11a7a974a693 (127.19.74.194:34653)
I20260806 22:42:35.100298 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:43029
--local_ip_for_outbound_sockets=127.19.74.195
--tserver_master_addrs=127.19.74.254:42469
--webserver_port=41261
--webserver_interface=127.19.74.195
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab
--principal=oryx/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857,127.19.74.252:43901 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:35.185235 27208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:35.185439 27208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:35.185463 27208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:35.186642 27208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:35.186685 27208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:35.186749 27208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:35.188259 27208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab
--principal=oryx/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:43029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=41261
--tserver_master_addrs=127.19.74.254:42469,127.19.74.253:34857,127.19.74.252:43901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:35.188513 27208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:35.188768 27208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:35.193809 27216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:35.193821 27214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:35.193825 27213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:35.194036 27208 server_base.cc:1034] running on GCE node
I20260806 22:42:35.194249 27208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:35.194541 27208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:35.195725 27208 hybrid_clock.cc:648] HybridClock initialized: now 1786056155195732 us; error 51 us; skew 500 ppm
Aug 06 22:42:35 dist-test-slave-pgkr krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:35.198167 27208 init.cc:377] Logged in from keytab as oryx/127.19.74.195@KRBTEST.COM (short username oryx)
I20260806 22:42:35.198755 27208 webserver.cc:460] Webserver started at http://127.19.74.195:41261/ using document root <none> and password file <none>
I20260806 22:42:35.199018 27208 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:35.199100 27208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:35.200451 27208 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:35.201059 27223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:35.201231 27208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:35.201323 27208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "88a22fda19bd44f68b6c7b93bd2b74c4"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-pgkr"
server_key: "5e5fccfe7ded21c6baaeccbc062d1a1d"
server_key_iv: "b1aad660d49d19e244cbafa4817089bc"
server_key_version: "encryptionkey@0"
I20260806 22:42:35.201680 27208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:35.225689 27208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:35.227761 27208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:35.227939 27208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:35.228186 27208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:35.228507 27208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:35.228564 27208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:35.228662 27208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:35.228688 27208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:35.234668 27208 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:43029
I20260806 22:42:35.234717 27339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:43029 every 8 connection(s)
I20260806 22:42:35.235102 27208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:35.236398 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 27208
Aug 06 22:42:35 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.253@KRBTEST.COM
I20260806 22:42:35.248406 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.237536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43804 (local address 127.19.74.252:43901)
0806 22:42:35.238177 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.238179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.238187 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:35.238209 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.238210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.238223 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.238920 (+ 697us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.239186 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.239958 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.241161 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.241461 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.241517 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.243641 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.243642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.243643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.243650 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.246811 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.247179 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.247180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.247180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.247264 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.247481 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.247482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.247483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.247635 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.247736 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.248159 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.248247 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":66,"threads_started":1}
Aug 06 22:42:35 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.252@KRBTEST.COM
Aug 06 22:42:35 dist-test-slave-pgkr krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:35.252067 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.237532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39768 (local address 127.19.74.253:34857)
0806 22:42:35.238120 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.238122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.238438 (+ 316us) server_negotiation.cc:407] Connection header received
0806 22:42:35.238465 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.238466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.238479 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.238542 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.239306 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.240027 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.243973 (+ 3946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.244288 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.244332 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.246860 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.246861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.246862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.246869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.249765 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.250343 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.250345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.250345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.250437 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.250823 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.250825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.250825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.251017 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.251178 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.251771 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.251916 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":52,"threads_started":1}
I20260806 22:42:35.254171 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.236004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:57815 (local address 127.19.74.252:43901)
0806 22:42:35.236104 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.236106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.236507 (+ 401us) server_negotiation.cc:407] Connection header received
0806 22:42:35.237945 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.237946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.237974 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.238019 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.239108 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.239883 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.241432 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.241735 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.241788 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.250537 (+ 8749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.250538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.250538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.250545 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.252455 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.253000 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.253001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.253001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.253063 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.253351 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.253352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.253353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.253501 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.253569 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.253950 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.254044 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":71}
I20260806 22:42:35.254423 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.235893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.19.74.195:57815)
0806 22:42:35.236452 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.236461 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.237796 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.238114 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.238115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.238691 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:35.238996 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.239002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.240002 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.240004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.241331 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.241334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.241855 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.241856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.242001 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.243334 (+ 1333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:35.243339 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:35.250434 (+ 7095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:35.252551 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.252560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.252563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.252915 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.253150 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.253151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.253151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.253277 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.253596 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:35.253603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:35.253778 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.253913 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.254118 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":105,"spinlock_wait_cycles":9600,"thread_start_us":104,"threads_started":1}
I20260806 22:42:35.254717 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.235893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.19.74.195:59659)
0806 22:42:35.236212 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.236261 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.237793 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.238114 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.238115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.238697 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:35.238996 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.239002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.240008 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.240009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.241331 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.241334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.241853 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.241854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.242001 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.242589 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:35.242600 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:35.244858 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:35.253521 (+ 8663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.253522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.253522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.253785 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.253996 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.253997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.253998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.254056 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.254344 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:35.254345 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:35.254386 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.254447 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.254571 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":52,"thread_start_us":97,"threads_started":1}
I20260806 22:42:35.254962 27340 heartbeater.cc:347] Connected to a master server at 127.19.74.252:43901
I20260806 22:42:35.255069 27340 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:35.255081 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.236151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:59659 (local address 127.19.74.253:34857)
0806 22:42:35.236205 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.236207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.236361 (+ 154us) server_negotiation.cc:407] Connection header received
0806 22:42:35.237945 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.237946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.237958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.238008 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.239135 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.239881 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.241432 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.241725 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.241786 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.249949 (+ 8163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.249951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.249951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.249959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.253454 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.253863 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.253864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.253865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.253922 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.254124 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.254125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.254125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.254253 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.254320 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.254869 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.254998 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:42:35.255275 27340 heartbeater.cc:511] Master 127.19.74.252:43901 requested a full tablet report, sending...
I20260806 22:42:35.255344 27356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.237547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.19.74.254:42469)
0806 22:42:35.237706 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.237708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.238416 (+ 708us) server_negotiation.cc:407] Connection header received
0806 22:42:35.240266 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.240267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.240282 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.240352 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.241026 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.241772 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.243618 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.243917 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.243980 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.247164 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.247166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.247166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.247174 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.253666 (+ 6492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.254082 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.254083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.254084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.254223 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.254434 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.254435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.254435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.254672 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.254782 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.255164 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.255248 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260806 22:42:35.255617 27342 heartbeater.cc:347] Connected to a master server at 127.19.74.253:34857
I20260806 22:42:35.255671 27342 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:35.255733 27342 heartbeater.cc:511] Master 127.19.74.253:34857 requested a full tablet report, sending...
I20260806 22:42:35.255721 26870 ts_manager.cc:194] Registered new tserver with Master: 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029)
I20260806 22:42:35.256011 26683 ts_manager.cc:194] Registered new tserver with Master: 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029)
I20260806 22:42:35.257051 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.235891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:38017)
0806 22:42:35.236249 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.236262 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.237782 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.238099 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.238103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.242273 (+ 4170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:35.242355 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.242357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.244948 (+ 2591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.244949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.245918 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.245920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.246905 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.246906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.246980 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.247827 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:35.247832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:35.251383 (+ 3551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:35.253498 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.253499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.253499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.255520 (+ 2021us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.256340 (+ 820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:35.256340 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:35.256341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:35.256436 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:35.256767 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:35.256767 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:35.256807 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.256868 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.256942 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":2,"thread_start_us":122,"threads_started":1}
I20260806 22:42:35.257169 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.236182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:38017 (local address 127.19.74.254:42469)
0806 22:42:35.236232 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.236234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.236386 (+ 152us) server_negotiation.cc:407] Connection header received
0806 22:42:35.237885 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.237886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.237899 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.237965 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.242425 (+ 4460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.243210 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.245985 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.246474 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.246573 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.251482 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.251484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.251484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.251492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.253420 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.256190 (+ 2770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.256191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.256191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.256254 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.256512 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.256513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.256513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.256671 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.256726 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.256953 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.257038 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:35.257467 27341 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42469
I20260806 22:42:35.257517 27341 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:35.257588 27341 heartbeater.cc:511] Master 127.19.74.254:42469 requested a full tablet report, sending...
I20260806 22:42:35.257836 26061 ts_manager.cc:194] Re-registered known tserver with Master: 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029)
I20260806 22:42:35.258297 26061 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:38017
I20260806 22:42:35.257342 26062 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:35.259344 26062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:35.267220 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.263095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:34653 (local address 127.0.0.1:54776)
0806 22:42:35.263235 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.263265 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.263304 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.263899 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.263900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.263901 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:35.263933 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.263934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.264965 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.264966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.266089 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.266092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.266986 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.266987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.267047 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.267050 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.267098 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.267138 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 22:42:35.267499 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.263180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.19.74.194:34653)
0806 22:42:35.263686 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.263691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.263701 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:35.263739 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.263742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.263758 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.263899 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:35.264016 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.264884 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.266179 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.266891 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.266975 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.267358 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.267392 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.267434 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":110,"threads_started":1}
I20260806 22:42:35.267638 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.263128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:37361 (local address 127.0.0.1:56268)
0806 22:42:35.263280 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.263291 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.263344 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.263785 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.263786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.263788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:35.263842 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.263844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.265232 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.265232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.266346 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.266349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.267388 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.267390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.267454 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.267458 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.267491 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.267538 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":39,"thread_start_us":95,"threads_started":1}
I20260806 22:42:35.267877 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.263408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.19.74.193:37361)
0806 22:42:35.263570 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.263575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.263586 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:35.263645 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.263647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.263665 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.263919 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:35.263977 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.265136 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.266426 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.267180 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.267630 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.267722 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.267749 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.267792 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260806 22:42:35.267968 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.263535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.19.74.195:43029)
0806 22:42:35.264020 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.264024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.264032 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:35.264064 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.264066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.264079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.264180 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:35.264303 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.265303 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.266536 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.267215 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.267279 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.267824 (+ 545us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.267848 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.267887 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":91,"threads_started":1}
I20260806 22:42:35.268110 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.263331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43029 (local address 127.0.0.1:45768)
0806 22:42:35.263443 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.263450 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.263500 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.264189 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.264190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.264192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:35.264226 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.264228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.265383 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.265384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.266452 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.266455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.267441 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.267442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.267500 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.267502 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.267541 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.267573 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260806 22:42:35.268285 27133 tablet_service.cc:1467] Processing CreateTablet for tablet 9034dbcdddf54036a75adeec38e65cbe (DEFAULT_TABLE table=test-table [id=596c042a4490438fab85e089ce626ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:35.268719 27133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9034dbcdddf54036a75adeec38e65cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:35.268821 27273 tablet_service.cc:1467] Processing CreateTablet for tablet 9034dbcdddf54036a75adeec38e65cbe (DEFAULT_TABLE table=test-table [id=596c042a4490438fab85e089ce626ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:35.269186 27273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9034dbcdddf54036a75adeec38e65cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:35.272691 27367 tablet_bootstrap.cc:492] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Bootstrap starting.
I20260806 22:42:35.273324 27367 tablet_bootstrap.cc:654] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:35.273358 26991 tablet_service.cc:1467] Processing CreateTablet for tablet 9034dbcdddf54036a75adeec38e65cbe (DEFAULT_TABLE table=test-table [id=596c042a4490438fab85e089ce626ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:35.273675 27367 log.cc:824] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:35.273783 26991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9034dbcdddf54036a75adeec38e65cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:35.274066 27366 tablet_bootstrap.cc:492] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Bootstrap starting.
I20260806 22:42:35.274530 27367 tablet_bootstrap.cc:492] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: No bootstrap required, opened a new log
I20260806 22:42:35.274605 27367 ts_tablet_manager.cc:1397] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:35.274796 27366 tablet_bootstrap.cc:654] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:35.275104 27366 log.cc:824] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:35.275935 27366 tablet_bootstrap.cc:492] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: No bootstrap required, opened a new log
I20260806 22:42:35.276006 27366 ts_tablet_manager.cc:1397] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:42:35.276777 27367 raft_consensus.cc:348] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.276955 27367 raft_consensus.cc:374] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:35.276995 27367 raft_consensus.cc:729] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b18d85f26ca4753b60c11a7a974a693, State: Initialized, Role: FOLLOWER
I20260806 22:42:35.277149 27367 consensus_queue.cc:260] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.277426 27367 ts_tablet_manager.cc:1428] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:42:35.277451 27371 tablet_bootstrap.cc:492] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Bootstrap starting.
I20260806 22:42:35.277842 27202 heartbeater.cc:503] Master 127.19.74.254:42469 was elected leader, sending a full tablet report...
I20260806 22:42:35.278124 27371 tablet_bootstrap.cc:654] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:35.278505 27371 log.cc:824] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:35.278580 27366 raft_consensus.cc:348] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.278793 27366 raft_consensus.cc:374] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:35.278846 27366 raft_consensus.cc:729] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a22fda19bd44f68b6c7b93bd2b74c4, State: Initialized, Role: FOLLOWER
I20260806 22:42:35.278990 27366 consensus_queue.cc:260] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.279279 27366 ts_tablet_manager.cc:1428] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 22:42:35.279321 27341 heartbeater.cc:503] Master 127.19.74.254:42469 was elected leader, sending a full tablet report...
I20260806 22:42:35.280450 27371 tablet_bootstrap.cc:492] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: No bootstrap required, opened a new log
I20260806 22:42:35.280530 27371 ts_tablet_manager.cc:1397] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:42:35.282452 27371 raft_consensus.cc:348] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.282603 27371 raft_consensus.cc:374] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:35.282663 27371 raft_consensus.cc:729] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df6c1f5bac54740a8d25e6118c6cdac, State: Initialized, Role: FOLLOWER
I20260806 22:42:35.282784 27371 consensus_queue.cc:260] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.283059 27371 ts_tablet_manager.cc:1428] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:42:35.283084 27062 heartbeater.cc:503] Master 127.19.74.254:42469 was elected leader, sending a full tablet report...
W20260806 22:42:35.331022 27204 tablet.cc:2369] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:35.411130 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.408448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.0.0.1:46864)
0806 22:42:35.408582 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.408597 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.408686 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.408879 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.408879 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.408881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:35.408934 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.408935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.409542 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.409542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.410262 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.410264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.410930 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.410931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.410981 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.410985 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.411023 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.411063 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 22:42:35.411124 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.408598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46864 (local address 127.19.74.252:43901)
0806 22:42:35.408671 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.408673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.408684 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:35.408741 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.408742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.408768 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.408830 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:35.409000 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.409475 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.410360 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.410852 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.410921 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.410990 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.411031 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.411069 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:42:35.411625 26886 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260806 22:42:35.412659 27376 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9854708559ab44a9a626401c6b3a2499. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:35.412772 27376 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: This master's current role is: LEARNER
W20260806 22:42:35.445341 27064 tablet.cc:2369] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:35.485962 27343 tablet.cc:2369] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:35.596019 27372 raft_consensus.cc:482] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:35.596148 27372 raft_consensus.cc:504] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.596489 27372 leader_election.cc:288] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0df6c1f5bac54740a8d25e6118c6cdac (127.19.74.193:37361), 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029)
I20260806 22:42:35.599432 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.596711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:37361 (local address 127.19.74.194:52719)
0806 22:42:35.596772 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.596785 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.596856 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.597027 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.597028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.597030 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:35.597100 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.597102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.597807 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.597808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.598593 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.598595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.599161 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.599162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.599257 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.599262 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.599311 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.599351 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:42:35.599454 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.596825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:52719 (local address 127.19.74.193:37361)
0806 22:42:35.596880 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.596882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.596895 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:42:35.596928 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.596929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.596940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.597011 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:35.597189 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.597721 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.598717 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.599092 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.599144 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.599151 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.599328 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.599368 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:42:35.599551 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.596922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:38913 (local address 127.19.74.195:43029)
0806 22:42:35.596999 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.597001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.597010 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:35.597074 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.597075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.597085 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.597151 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:35.597304 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.597781 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.598716 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.599220 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.599295 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.599303 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.599445 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.599488 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:42:35.599756 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.596820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43029 (local address 127.19.74.194:38913)
0806 22:42:35.596966 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:35.596976 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:35.597019 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:35.597176 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:35.597177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:35.597179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:35.597234 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.597236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.597872 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.597873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.598596 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:35.598598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.599306 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.599307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:35.599366 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.599370 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:35.599428 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:42:35.599464 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260806 22:42:35.599851 27013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9034dbcdddf54036a75adeec38e65cbe" candidate_uuid: "0b18d85f26ca4753b60c11a7a974a693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" is_pre_election: true
I20260806 22:42:35.600045 27013 raft_consensus.cc:2454] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b18d85f26ca4753b60c11a7a974a693 in term 0.
I20260806 22:42:35.600136 27294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9034dbcdddf54036a75adeec38e65cbe" candidate_uuid: "0b18d85f26ca4753b60c11a7a974a693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" is_pre_election: true
I20260806 22:42:35.600281 27294 raft_consensus.cc:2454] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b18d85f26ca4753b60c11a7a974a693 in term 0.
I20260806 22:42:35.600291 27087 leader_election.cc:302] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b18d85f26ca4753b60c11a7a974a693, 0df6c1f5bac54740a8d25e6118c6cdac; no voters:
I20260806 22:42:35.600423 27372 raft_consensus.cc:2781] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:35.600482 27372 raft_consensus.cc:482] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:35.600508 27372 raft_consensus.cc:3037] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:35.601233 27372 raft_consensus.cc:504] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.601366 27372 leader_election.cc:288] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [CANDIDATE]: Term 1 election: Requested vote from peers 0df6c1f5bac54740a8d25e6118c6cdac (127.19.74.193:37361), 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029)
I20260806 22:42:35.601502 27013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9034dbcdddf54036a75adeec38e65cbe" candidate_uuid: "0b18d85f26ca4753b60c11a7a974a693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df6c1f5bac54740a8d25e6118c6cdac"
I20260806 22:42:35.601527 27294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9034dbcdddf54036a75adeec38e65cbe" candidate_uuid: "0b18d85f26ca4753b60c11a7a974a693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4"
I20260806 22:42:35.601581 27013 raft_consensus.cc:3037] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:35.601627 27294 raft_consensus.cc:3037] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:35.602276 27013 raft_consensus.cc:2454] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b18d85f26ca4753b60c11a7a974a693 in term 1.
I20260806 22:42:35.602450 27294 raft_consensus.cc:2454] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b18d85f26ca4753b60c11a7a974a693 in term 1.
I20260806 22:42:35.602450 27087 leader_election.cc:302] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b18d85f26ca4753b60c11a7a974a693, 0df6c1f5bac54740a8d25e6118c6cdac; no voters:
I20260806 22:42:35.602541 27372 raft_consensus.cc:2781] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:35.602674 27372 raft_consensus.cc:686] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 1 LEADER]: Becoming Leader. State: Replica: 0b18d85f26ca4753b60c11a7a974a693, State: Running, Role: LEADER
I20260806 22:42:35.602769 27372 consensus_queue.cc:237] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:35.603639 26061 catalog_manager.cc:5495] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b18d85f26ca4753b60c11a7a974a693 (127.19.74.194). New cstate: current_term: 1 leader_uuid: "0b18d85f26ca4753b60c11a7a974a693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:35.604350 26838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: NON_VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: true } } }
I20260806 22:42:35.604444 26838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:35.605264 26838 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: attempting to promote NON_VOTER e36883e05a7e4f5ebd44e7415ac5c536 to VOTER
I20260806 22:42:35.605759 26838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } }
I20260806 22:42:35.606211 26693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Refusing update from remote peer 9854708559ab44a9a626401c6b3a2499: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 22:42:35.606256 26886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 LEARNER]: Refusing update from remote peer 9854708559ab44a9a626401c6b3a2499: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 22:42:35.606469 27386 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 22:42:35.606570 26837 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 22:42:35.607121 27386 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:35.607254 26693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:35.607329 26886 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:35.607888 27386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:35.607966 27386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:35.607944 26837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:35.608007 26837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:35.608017 27386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:35.608094 27386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:35.608196 27388 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:35.608273 27388 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:35.608390 27376 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9854708559ab44a9a626401c6b3a2499" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:35.608508 27376 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:35.623113 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.621032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.19.74.194:34653)
0806 22:42:35.621095 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.621097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.621105 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:35.621145 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.621146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.621161 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.621221 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:35.621331 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.621785 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.622438 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.622606 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.622707 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.622837 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:35.622932 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:35.622991 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.623021 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.623037 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:42:35.625025 27013 raft_consensus.cc:1261] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 1 FOLLOWER]: Refusing update from remote peer 0b18d85f26ca4753b60c11a7a974a693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:35.625025 27294 raft_consensus.cc:1261] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 1 FOLLOWER]: Refusing update from remote peer 0b18d85f26ca4753b60c11a7a974a693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:35.625411 27382 consensus_queue.cc:1035] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:35.625573 27372 consensus_queue.cc:1035] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:35.629241 27392 mvcc.cc:205] Tried to move back new op lower bound from 7315686013436997632 to 7315686013349601280. Current Snapshot: MvccSnapshot[applied={T|T < 7315686013436997632}]
I20260806 22:42:35.630766 27391 mvcc.cc:205] Tried to move back new op lower bound from 7315686013436997632 to 7315686013349601280. Current Snapshot: MvccSnapshot[applied={T|T < 7315686013436997632 or (T in {7315686013436997632})}]
I20260806 22:42:35.630857 26061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:35.630945 26061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:35.632411 26061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9854708559ab44a9a626401c6b3a2499: Sending DeleteTablet for 3 replicas of tablet 9034dbcdddf54036a75adeec38e65cbe
I20260806 22:42:35.632730 26991 tablet_service.cc:1514] Processing DeleteTablet for tablet 9034dbcdddf54036a75adeec38e65cbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:35 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:56268
I20260806 22:42:35.632751 27273 tablet_service.cc:1514] Processing DeleteTablet for tablet 9034dbcdddf54036a75adeec38e65cbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:35 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:45768
I20260806 22:42:35.633139 27400 tablet_replica.cc:330] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: stopping tablet replica
I20260806 22:42:35.633376 27133 tablet_service.cc:1514] Processing DeleteTablet for tablet 9034dbcdddf54036a75adeec38e65cbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:35 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:54776
I20260806 22:42:35.633291 27400 raft_consensus.cc:2229] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:35.633756 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 26030
I20260806 22:42:35.633795 27400 raft_consensus.cc:2258] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:35.634224 27400 ts_tablet_manager.cc:1905] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:35.636322 27400 ts_tablet_manager.cc:1918] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:35.636399 27400 log.cc:1196] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/9034dbcdddf54036a75adeec38e65cbe
I20260806 22:42:35.636744 27400 ts_tablet_manager.cc:1939] T 9034dbcdddf54036a75adeec38e65cbe P 88a22fda19bd44f68b6c7b93bd2b74c4: Deleting consensus metadata
I20260806 22:42:35.637146 27402 tablet_replica.cc:330] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: stopping tablet replica
I20260806 22:42:35.637274 27402 raft_consensus.cc:2229] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:35.637369 27402 raft_consensus.cc:2258] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:35.637833 27402 ts_tablet_manager.cc:1905] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:35.639536 27402 ts_tablet_manager.cc:1918] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:35.639652 27402 log.cc:1196] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/9034dbcdddf54036a75adeec38e65cbe
I20260806 22:42:35.639986 27402 ts_tablet_manager.cc:1939] T 9034dbcdddf54036a75adeec38e65cbe P 0b18d85f26ca4753b60c11a7a974a693: Deleting consensus metadata
I20260806 22:42:35.640355 27399 tablet_replica.cc:330] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: stopping tablet replica
I20260806 22:42:35.640470 27399 raft_consensus.cc:2229] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:35.640568 27399 raft_consensus.cc:2258] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:35.641574 27399 ts_tablet_manager.cc:1905] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:35.643383 27399 ts_tablet_manager.cc:1918] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:35.643457 27399 log.cc:1196] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/9034dbcdddf54036a75adeec38e65cbe
I20260806 22:42:35.643743 27399 ts_tablet_manager.cc:1939] T 9034dbcdddf54036a75adeec38e65cbe P 0df6c1f5bac54740a8d25e6118c6cdac: Deleting consensus metadata
W20260806 22:42:35.643740 27224 connection.cc:535] server connection from 127.0.0.1:45768 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:42:35.644078 27087 connection.cc:535] server connection from 127.0.0.1:54776 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:42:35.644233 26945 connection.cc:441] server connection from 127.0.0.1:56268 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56268 (request call id 1) could send its response
I20260806 22:42:35.646858 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.633544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.19.74.252:43901)
0806 22:42:35.633624 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.633626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.637586 (+ 3960us) server_negotiation.cc:407] Connection header received
0806 22:42:35.637702 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.637703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.637714 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.637744 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.638510 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.639216 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.640712 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.640916 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.640949 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.642774 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.642776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.642776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.642783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.644630 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.645018 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.645019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.645020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.645082 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.645314 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.645315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.645315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.645447 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.645560 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.646680 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.646754 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:42:35.654207 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.646473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46882 (local address 127.19.74.252:43901)
0806 22:42:35.646513 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.646515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.647127 (+ 612us) server_negotiation.cc:407] Connection header received
0806 22:42:35.647144 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.647145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.647156 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.647208 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.648122 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.648563 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.649280 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.649488 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.649561 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.651011 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.651013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.651013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.651020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.652796 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.653193 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.653194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.653195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.653266 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.653516 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.653518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.653518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.653651 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.653758 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.654037 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.654120 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:35.654803 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:35.646971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59042 (local address 127.19.74.253:34857)
0806 22:42:35.647016 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:35.647018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:35.647187 (+ 169us) server_negotiation.cc:407] Connection header received
0806 22:42:35.647304 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:35.647305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:35.647317 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:35.647363 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:35.648138 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.648828 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.649869 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:35.650170 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:35.650213 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:35.651687 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:35.651688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:35.651689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:35.651696 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:35.653515 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.653856 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.653857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.653858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.653920 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:35.654188 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:35.654189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:35.654189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:35.654336 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:35.654415 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:35.654587 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:42:35.654747 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:36.630020 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.629741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:55921)
0806 22:42:36.629884 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.629957 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
W20260806 22:42:36.630349 27202 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:42469 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:36.630837 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.630532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:42883)
0806 22:42:36.630696 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.630771 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
W20260806 22:42:36.631137 27062 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:42469 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:36.631692 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.631416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:48371)
0806 22:42:36.631553 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.631646 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":49,"threads_started":1}
W20260806 22:42:36.631876 27341 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:42469 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:37.133570 27412 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9854708559ab44a9a626401c6b3a2499)
I20260806 22:42:37.133598 27413 raft_consensus.cc:482] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9854708559ab44a9a626401c6b3a2499)
I20260806 22:42:37.133733 27413 raft_consensus.cc:504] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } }
I20260806 22:42:37.133733 27412 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } }
I20260806 22:42:37.134016 27413 leader_election.cc:288] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469), ff7bf2742f8443c58614336a2c3ea399 (127.19.74.253:34857)
I20260806 22:42:37.134021 27412 leader_election.cc:288] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469), e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252:43901)
I20260806 22:42:37.134326 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.134109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34176)
0806 22:42:37.134230 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.134281 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260806 22:42:37.134413 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.134104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34172)
0806 22:42:37.134333 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.134373 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":47,"threads_started":1}
W20260806 22:42:37.134454 26861 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
W20260806 22:42:37.134543 26668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:37.134992 27416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.134878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34178)
0806 22:42:37.134918 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.134954 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:42:37.135084 26668 leader_election.cc:334] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469): Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:37.135215 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.135003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34188)
0806 22:42:37.135149 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.135183 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
W20260806 22:42:37.135309 26861 leader_election.cc:334] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469): Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:37.138073 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.134222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:43901 (local address 127.0.0.1:46896)
0806 22:42:37.134456 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.134467 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.134522 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.134751 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.134753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.134755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:37.134821 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.134823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.135850 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.135851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.137079 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.137082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.137826 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.137827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.137899 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.137904 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.137959 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.138008 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":42,"threads_started":1}
I20260806 22:42:37.138132 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.134333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46896 (local address 127.19.74.252:43901)
0806 22:42:37.134583 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.134585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.134592 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:37.134631 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.134632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.134643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.134723 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:37.134986 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.135771 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.137162 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.137756 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.137826 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.137841 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.137980 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.138044 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":48,"threads_started":1}
I20260806 22:42:37.138178 27418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.134445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.19.74.253:34857)
0806 22:42:37.134823 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.134825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.134832 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:37.134852 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.134852 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.134862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.134938 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:37.135134 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.135891 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.137249 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.137826 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.137889 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.137906 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.138036 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.138080 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":42,"threads_started":1}
I20260806 22:42:37.138326 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.134374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:34857 (local address 127.0.0.1:59044)
0806 22:42:37.134430 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.134439 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.134489 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.134943 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.134944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.134946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:37.134994 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.134996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.135966 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.135967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.137166 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.137170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.137896 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.137897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.137962 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.137967 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.138018 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.138058 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:42:37.138406 26886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff7bf2742f8443c58614336a2c3ea399" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" is_pre_election: true
I20260806 22:42:37.138545 26886 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7bf2742f8443c58614336a2c3ea399 in term 1.
I20260806 22:42:37.138569 26693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff7bf2742f8443c58614336a2c3ea399" is_pre_election: true
I20260806 22:42:37.138676 26693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36883e05a7e4f5ebd44e7415ac5c536 in term 1.
I20260806 22:42:37.138734 26667 leader_election.cc:302] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e36883e05a7e4f5ebd44e7415ac5c536, ff7bf2742f8443c58614336a2c3ea399; no voters: 9854708559ab44a9a626401c6b3a2499
I20260806 22:42:37.138811 26863 leader_election.cc:302] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e36883e05a7e4f5ebd44e7415ac5c536, ff7bf2742f8443c58614336a2c3ea399; no voters: 9854708559ab44a9a626401c6b3a2499
I20260806 22:42:37.138835 27412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:42:37.138867 27412 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9854708559ab44a9a626401c6b3a2499)
I20260806 22:42:37.138887 27412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:42:37.138918 27413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:42:37.138957 27413 raft_consensus.cc:482] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9854708559ab44a9a626401c6b3a2499)
I20260806 22:42:37.138981 27413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:42:37.139810 27413 raft_consensus.cc:504] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } }
I20260806 22:42:37.139810 27412 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } }
I20260806 22:42:37.139945 27413 leader_election.cc:288] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [CANDIDATE]: Term 2 election: Requested vote from peers 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469), ff7bf2742f8443c58614336a2c3ea399 (127.19.74.253:34857)
I20260806 22:42:37.139945 27412 leader_election.cc:288] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 2 election: Requested vote from peers 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469), e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252:43901)
I20260806 22:42:37.140200 26886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff7bf2742f8443c58614336a2c3ea399" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e36883e05a7e4f5ebd44e7415ac5c536"
I20260806 22:42:37.140236 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.140070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34194)
0806 22:42:37.140117 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.140167 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:42:37.140182 26693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff7bf2742f8443c58614336a2c3ea399"
I20260806 22:42:37.140311 26886 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7bf2742f8443c58614336a2c3ea399 in current term 2: Already voted for candidate e36883e05a7e4f5ebd44e7415ac5c536 in this term.
I20260806 22:42:37.140327 26693 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e36883e05a7e4f5ebd44e7415ac5c536 in current term 2: Already voted for candidate ff7bf2742f8443c58614336a2c3ea399 in this term.
I20260806 22:42:37.140362 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.140124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34206)
0806 22:42:37.140287 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.140319 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 22:42:37.140623 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.140516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34208)
0806 22:42:37.140545 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.140572 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 22:42:37.140676 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.140578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34224)
0806 22:42:37.140625 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.140651 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:42:37.140729 26668 leader_election.cc:334] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469): Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:37.140777 26668 leader_election.cc:302] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff7bf2742f8443c58614336a2c3ea399; no voters: 9854708559ab44a9a626401c6b3a2499, e36883e05a7e4f5ebd44e7415ac5c536
I20260806 22:42:37.140863 27412 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260806 22:42:37.140909 26861 leader_election.cc:334] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469): Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:37.140952 26861 leader_election.cc:302] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e36883e05a7e4f5ebd44e7415ac5c536; no voters: 9854708559ab44a9a626401c6b3a2499, ff7bf2742f8443c58614336a2c3ea399
I20260806 22:42:37.141028 27413 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 22:42:37.631098 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.630751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:39507)
0806 22:42:37.630973 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.631035 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":125,"threads_started":1}
I20260806 22:42:37.631700 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.631487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:34113)
0806 22:42:37.631615 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.631657 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260806 22:42:37.632493 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.632169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:40781)
0806 22:42:37.632410 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.632450 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":116,"threads_started":1}
I20260806 22:42:38.632035 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.631657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:43509)
0806 22:42:38.631817 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.631975 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260806 22:42:38.632495 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.632302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:38769)
0806 22:42:38.632431 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.632460 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260806 22:42:38.633095 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.632871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:39485)
0806 22:42:38.633011 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.633050 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260806 22:42:38.723146 27428 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:38.723230 27428 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } }
I20260806 22:42:38.723399 27428 leader_election.cc:288] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469), e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252:43901)
I20260806 22:42:38.723616 26886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff7bf2742f8443c58614336a2c3ea399" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" is_pre_election: true
I20260806 22:42:38.723706 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.723514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34460)
0806 22:42:38.723612 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.723666 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":44,"threads_started":1}
I20260806 22:42:38.723743 26886 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff7bf2742f8443c58614336a2c3ea399 in term 2.
I20260806 22:42:38.723912 26667 leader_election.cc:302] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e36883e05a7e4f5ebd44e7415ac5c536, ff7bf2742f8443c58614336a2c3ea399; no voters:
I20260806 22:42:38.724011 27428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 22:42:38.724050 27428 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:38.724090 27428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:42:38.724186 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.724084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34468)
0806 22:42:38.724142 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.724163 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 22:42:38.724280 26668 leader_election.cc:334] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469): Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:38.724924 27428 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } }
I20260806 22:42:38.725061 27428 leader_election.cc:288] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 3 election: Requested vote from peers 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469), e36883e05a7e4f5ebd44e7415ac5c536 (127.19.74.252:43901)
I20260806 22:42:38.725210 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.725124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34480)
0806 22:42:38.725159 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.725194 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 22:42:38.725191 26886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff7bf2742f8443c58614336a2c3ea399" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e36883e05a7e4f5ebd44e7415ac5c536"
I20260806 22:42:38.725260 26886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:42:38.725538 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.725447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34484)
0806 22:42:38.725494 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.725516 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:42:38.725666 26668 leader_election.cc:334] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469): Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
I20260806 22:42:38.725994 26886 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7bf2742f8443c58614336a2c3ea399 in term 3.
I20260806 22:42:38.726156 26667 leader_election.cc:302] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e36883e05a7e4f5ebd44e7415ac5c536, ff7bf2742f8443c58614336a2c3ea399; no voters: 9854708559ab44a9a626401c6b3a2499
I20260806 22:42:38.726339 27428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:42:38.726464 27428 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [term 3 LEADER]: Becoming Leader. State: Replica: ff7bf2742f8443c58614336a2c3ea399, State: Running, Role: LEADER
I20260806 22:42:38.726557 27428 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } }
I20260806 22:42:38.726936 27431 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff7bf2742f8443c58614336a2c3ea399. Latest consensus state: current_term: 3 leader_uuid: "ff7bf2742f8443c58614336a2c3ea399" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:38.727041 27431 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:38.727236 27433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:38.727663 27433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:38.727744 27433 catalog_manager.cc:1259] Loaded cluster ID: dc5f99a5646847c789e5f4e81ec45ea9
I20260806 22:42:38.727777 27433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:38.727980 27433 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:38.728084 27433 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Loaded TSK: 0
I20260806 22:42:38.728243 27433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:42:38.804775 26682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:38.806509 26682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:38.807554 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.807437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34502)
0806 22:42:38.807490 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.807518 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:42:38.807617 26886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [term 3 FOLLOWER]: Refusing update from remote peer ff7bf2742f8443c58614336a2c3ea399: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 22:42:38.807854 27428 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 22:42:38.807922 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.807787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.0.0.1:34510)
0806 22:42:38.807840 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.807873 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:42:38.808036 26668 consensus_peers.cc:487] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 -> Peer 9854708559ab44a9a626401c6b3a2499 (127.19.74.254:42469): Couldn't send request to peer 9854708559ab44a9a626401c6b3a2499. Status: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:42:38.808552 27435 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff7bf2742f8443c58614336a2c3ea399. Latest consensus state: current_term: 3 leader_uuid: "ff7bf2742f8443c58614336a2c3ea399" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:38.808671 27435 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:38.809194 27428 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ff7bf2742f8443c58614336a2c3ea399" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:38.809301 27428 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:38.809451 27435 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ff7bf2742f8443c58614336a2c3ea399" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9854708559ab44a9a626401c6b3a2499" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42469 } } peers { permanent_uuid: "ff7bf2742f8443c58614336a2c3ea399" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "e36883e05a7e4f5ebd44e7415ac5c536" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 43901 } attrs { promote: false } } }
I20260806 22:42:38.809588 27435 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36883e05a7e4f5ebd44e7415ac5c536 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:38.814556 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.811211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.19.74.193:37361)
0806 22:42:38.811456 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:38.811458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:38.811466 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:38.811489 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:38.811490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:38.811572 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:38.811811 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:38.811876 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.812396 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.813654 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.814190 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.814256 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.814285 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:38.814432 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:42:38.814480 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":44,"threads_started":1}
I20260806 22:42:38.814541 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.811022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:37361 (local address 127.0.0.1:35882)
0806 22:42:38.811052 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.811062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:38.811123 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:38.811684 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:38.811685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:38.811686 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:38.811734 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.811735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.812485 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.812486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.813582 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.813584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.814290 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.814291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.814353 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.814358 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:38.814412 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:42:38.814461 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260806 22:42:38.814741 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.811277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.19.74.194:34653)
0806 22:42:38.811416 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:38.811436 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:38.811446 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:38.811475 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:38.811476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:38.811488 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:38.811567 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:38.811812 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.812587 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.813636 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.814189 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.814256 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.814284 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:38.814570 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:42:38.814652 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260806 22:42:38.814741 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.811305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35454 (local address 127.19.74.195:43029)
0806 22:42:38.811439 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:38.811441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:38.811695 (+ 254us) server_negotiation.cc:407] Connection header received
0806 22:42:38.811763 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:38.811764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:38.811776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:38.811866 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:38.811968 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.812679 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.813866 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.814248 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.814331 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.814342 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:38.814582 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 22:42:38.814633 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260806 22:42:38.814754 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.811120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43029 (local address 127.0.0.1:35454)
0806 22:42:38.811643 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.811654 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:38.811703 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:38.811880 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:38.811881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:38.811882 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:38.811914 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.811915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.812757 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.812758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.813789 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.813791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.814370 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.814371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.814426 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.814430 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:38.814471 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 22:42:38.814508 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":86,"threads_started":1}
I20260806 22:42:38.814961 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.811089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:34653 (local address 127.0.0.1:59822)
0806 22:42:38.811249 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.811258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:38.811302 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:38.811642 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:38.811643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:38.811645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:38.811704 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.811706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.812692 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.812693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.813540 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.813543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.814319 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.814320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.814386 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.814408 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:42:38.814459 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:42:38.814497 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260806 22:42:38.815152 27273 tablet_service.cc:1467] Processing CreateTablet for tablet a365b2f9840f4ddfaf8f8f1a18355dbb (DEFAULT_TABLE table=test-table [id=9987c640bb134d4eb5663d837c517764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:38.815196 27133 tablet_service.cc:1467] Processing CreateTablet for tablet a365b2f9840f4ddfaf8f8f1a18355dbb (DEFAULT_TABLE table=test-table [id=9987c640bb134d4eb5663d837c517764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:38.815305 27133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a365b2f9840f4ddfaf8f8f1a18355dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:38.815305 27273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a365b2f9840f4ddfaf8f8f1a18355dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:38.816907 27447 tablet_bootstrap.cc:492] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: Bootstrap starting.
I20260806 22:42:38.816955 26991 tablet_service.cc:1467] Processing CreateTablet for tablet a365b2f9840f4ddfaf8f8f1a18355dbb (DEFAULT_TABLE table=test-table [id=9987c640bb134d4eb5663d837c517764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:38.817081 26991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a365b2f9840f4ddfaf8f8f1a18355dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:38.817332 27447 tablet_bootstrap.cc:654] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:38.817624 27446 tablet_bootstrap.cc:492] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: Bootstrap starting.
I20260806 22:42:38.817984 27446 tablet_bootstrap.cc:654] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:38.818324 27450 tablet_bootstrap.cc:492] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac: Bootstrap starting.
I20260806 22:42:38.818493 27447 tablet_bootstrap.cc:492] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: No bootstrap required, opened a new log
I20260806 22:42:38.818598 27447 ts_tablet_manager.cc:1397] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:38.818809 27450 tablet_bootstrap.cc:654] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:38.818789 27447 raft_consensus.cc:348] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.818862 27447 raft_consensus.cc:374] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:38.818876 27447 raft_consensus.cc:729] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a22fda19bd44f68b6c7b93bd2b74c4, State: Initialized, Role: FOLLOWER
I20260806 22:42:38.818959 27447 consensus_queue.cc:260] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.819159 27447 ts_tablet_manager.cc:1428] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:38.819613 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.819341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:56669)
0806 22:42:38.819551 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.819584 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 22:42:38.819770 27342 heartbeater.cc:503] Master 127.19.74.253:34857 was elected leader, sending a full tablet report...
I20260806 22:42:38.819931 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.819838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:48989)
0806 22:42:38.819876 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.819905 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 22:42:38.820211 27446 tablet_bootstrap.cc:492] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: No bootstrap required, opened a new log
I20260806 22:42:38.820290 27446 ts_tablet_manager.cc:1397] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:42:38.820416 27446 raft_consensus.cc:348] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.820502 27446 raft_consensus.cc:374] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:38.820534 27446 raft_consensus.cc:729] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b18d85f26ca4753b60c11a7a974a693, State: Initialized, Role: FOLLOWER
I20260806 22:42:38.820586 27446 consensus_queue.cc:260] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.820747 27446 ts_tablet_manager.cc:1428] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:38.821043 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.820946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:43519)
0806 22:42:38.820984 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.821014 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:42:38.821266 27450 tablet_bootstrap.cc:492] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac: No bootstrap required, opened a new log
I20260806 22:42:38.821321 27450 ts_tablet_manager.cc:1397] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:42:38.821360 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.821255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:51265)
0806 22:42:38.821311 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.821337 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:42:38.821480 27450 raft_consensus.cc:348] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.821553 27450 raft_consensus.cc:374] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:38.821573 27450 raft_consensus.cc:729] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df6c1f5bac54740a8d25e6118c6cdac, State: Initialized, Role: FOLLOWER
I20260806 22:42:38.821619 27450 consensus_queue.cc:260] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.821806 27450 ts_tablet_manager.cc:1428] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:38.822121 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.821946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:47433)
0806 22:42:38.822054 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.822080 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 22:42:38.822431 27063 heartbeater.cc:503] Master 127.19.74.253:34857 was elected leader, sending a full tablet report...
I20260806 22:42:38.822476 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.822375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:33949)
0806 22:42:38.822413 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.822448 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 22:42:38.846800 27454 raft_consensus.cc:482] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:38.846900 27454 raft_consensus.cc:504] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.847289 27454 leader_election.cc:288] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b18d85f26ca4753b60c11a7a974a693 (127.19.74.194:34653), 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029)
I20260806 22:42:38.851217 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.847539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:43629 (local address 127.19.74.195:43029)
0806 22:42:38.847682 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:38.847684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:38.847690 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:42:38.847712 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:38.847713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:38.847720 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:38.847758 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:38.847987 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.848757 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.850266 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.850804 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.850878 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.850887 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:38.851102 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 22:42:38.851145 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 22:42:38.851253 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.847530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:32881 (local address 127.19.74.194:34653)
0806 22:42:38.847573 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:38.847575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:38.847829 (+ 254us) server_negotiation.cc:407] Connection header received
0806 22:42:38.847864 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:38.847865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:38.847872 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:38.847913 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:38.848064 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.848768 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.850266 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.850804 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.850878 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.850887 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:38.851136 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:42:38.851178 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:38.851244 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.847440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43029 (local address 127.19.74.193:43629)
0806 22:42:38.847499 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.847513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:38.847580 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:38.847806 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:38.847808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:38.847811 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:38.847913 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.847916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.848843 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.848843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.850164 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.850167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.850889 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.850890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.851000 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.851006 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:42:38.851077 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:42:38.851130 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":113}
I20260806 22:42:38.851259 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.847439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:34653 (local address 127.19.74.193:32881)
0806 22:42:38.847719 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.847728 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:38.847814 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:38.847929 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:38.847931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:38.847932 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:38.847990 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.847991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.848842 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.848843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.850164 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:38.850167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.850909 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.850910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:38.851002 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.851004 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:38.851077 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:42:38.851130 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":107,"thread_start_us":136,"threads_started":1}
I20260806 22:42:38.851536 27294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a365b2f9840f4ddfaf8f8f1a18355dbb" candidate_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" is_pre_election: true
I20260806 22:42:38.851612 27294 raft_consensus.cc:2454] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0df6c1f5bac54740a8d25e6118c6cdac in term 0.
I20260806 22:42:38.851644 27154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a365b2f9840f4ddfaf8f8f1a18355dbb" candidate_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18d85f26ca4753b60c11a7a974a693" is_pre_election: true
I20260806 22:42:38.851794 27154 raft_consensus.cc:2454] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0df6c1f5bac54740a8d25e6118c6cdac in term 0.
I20260806 22:42:38.851778 26945 leader_election.cc:302] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df6c1f5bac54740a8d25e6118c6cdac, 88a22fda19bd44f68b6c7b93bd2b74c4; no voters:
I20260806 22:42:38.851936 27454 raft_consensus.cc:2781] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:38.851974 27454 raft_consensus.cc:482] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:38.851994 27454 raft_consensus.cc:3037] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:38.852708 27454 raft_consensus.cc:504] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.852857 27454 leader_election.cc:288] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [CANDIDATE]: Term 1 election: Requested vote from peers 0b18d85f26ca4753b60c11a7a974a693 (127.19.74.194:34653), 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029)
I20260806 22:42:38.853022 27294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a365b2f9840f4ddfaf8f8f1a18355dbb" candidate_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4"
I20260806 22:42:38.853036 27154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a365b2f9840f4ddfaf8f8f1a18355dbb" candidate_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18d85f26ca4753b60c11a7a974a693"
I20260806 22:42:38.853134 27294 raft_consensus.cc:3037] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:38.853132 27154 raft_consensus.cc:3037] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:38.853753 27294 raft_consensus.cc:2454] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0df6c1f5bac54740a8d25e6118c6cdac in term 1.
I20260806 22:42:38.853790 27154 raft_consensus.cc:2454] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0df6c1f5bac54740a8d25e6118c6cdac in term 1.
I20260806 22:42:38.853971 26944 leader_election.cc:302] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b18d85f26ca4753b60c11a7a974a693, 0df6c1f5bac54740a8d25e6118c6cdac, 88a22fda19bd44f68b6c7b93bd2b74c4; no voters:
I20260806 22:42:38.854079 27454 raft_consensus.cc:2781] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:38.854238 27454 raft_consensus.cc:686] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 1 LEADER]: Becoming Leader. State: Replica: 0df6c1f5bac54740a8d25e6118c6cdac, State: Running, Role: LEADER
I20260806 22:42:38.854457 27454 consensus_queue.cc:237] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } }
I20260806 22:42:38.854533 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.854432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:58645)
0806 22:42:38.854476 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.854509 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:42:38.855195 26682 catalog_manager.cc:5495] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac reported cstate change: term changed from 0 to 1, leader changed from <none> to 0df6c1f5bac54740a8d25e6118c6cdac (127.19.74.193). New cstate: current_term: 1 leader_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df6c1f5bac54740a8d25e6118c6cdac" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 37361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:38.871995 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.869560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.19.74.193:37361)
0806 22:42:38.869634 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:38.869636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:38.869644 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:38.869684 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:38.869685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:38.869702 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:38.869778 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:38.869920 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.870361 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.871151 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:38.871334 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:38.871437 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:38.871613 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:38.871745 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:38.871837 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:38.871880 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:42:38.871900 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:42:38.873023 27294 raft_consensus.cc:1261] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 1 FOLLOWER]: Refusing update from remote peer 0df6c1f5bac54740a8d25e6118c6cdac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:38.873189 27154 raft_consensus.cc:1261] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 1 FOLLOWER]: Refusing update from remote peer 0df6c1f5bac54740a8d25e6118c6cdac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:38.873332 27456 consensus_queue.cc:1035] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a22fda19bd44f68b6c7b93bd2b74c4" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:38.873461 27454 consensus_queue.cc:1035] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b18d85f26ca4753b60c11a7a974a693" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:38.874280 27203 heartbeater.cc:503] Master 127.19.74.253:34857 was elected leader, sending a full tablet report...
I20260806 22:42:38.874646 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.874092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:50687)
0806 22:42:38.874539 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.874570 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428}
I20260806 22:42:38.874869 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.874492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:45569)
0806 22:42:38.874790 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.874826 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280}
I20260806 22:42:38.875494 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.875367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:45121)
0806 22:42:38.875443 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.875468 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:42:38.875535 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.875380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.194:56705)
0806 22:42:38.875476 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.875503 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 22:42:38.875977 27462 mvcc.cc:205] Tried to move back new op lower bound from 7315686026741886976 to 7315686026668613632. Current Snapshot: MvccSnapshot[applied={T|T < 7315686026741886976}]
I20260806 22:42:38.876145 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.876008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.193:59627)
0806 22:42:38.876089 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.876115 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 22:42:38.876704 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.876596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42469 (local address 127.19.74.195:37819)
0806 22:42:38.876648 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.876673 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:42:38.878372 26683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:38.878465 26683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:38.879992 26683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff7bf2742f8443c58614336a2c3ea399: Sending DeleteTablet for 3 replicas of tablet a365b2f9840f4ddfaf8f8f1a18355dbb
I20260806 22:42:38.880311 27273 tablet_service.cc:1514] Processing DeleteTablet for tablet a365b2f9840f4ddfaf8f8f1a18355dbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:38 UTC) from {username='oryx', principal='oryx/127.19.74.253@KRBTEST.COM'} at 127.0.0.1:35454
I20260806 22:42:38.880314 27133 tablet_service.cc:1514] Processing DeleteTablet for tablet a365b2f9840f4ddfaf8f8f1a18355dbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:38 UTC) from {username='oryx', principal='oryx/127.19.74.253@KRBTEST.COM'} at 127.0.0.1:59822
I20260806 22:42:38.880565 26991 tablet_service.cc:1514] Processing DeleteTablet for tablet a365b2f9840f4ddfaf8f8f1a18355dbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:38 UTC) from {username='oryx', principal='oryx/127.19.74.253@KRBTEST.COM'} at 127.0.0.1:35882
I20260806 22:42:38.880764 27470 tablet_replica.cc:330] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac: stopping tablet replica
I20260806 22:42:38.880820 27470 raft_consensus.cc:2229] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0df6c1f5bac54740a8d25e6118c6cdac [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:38.880844 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 26834
I20260806 22:42:38.883160 27468 tablet_replica.cc:330] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: stopping tablet replica
I20260806 22:42:38.883242 27468 raft_consensus.cc:2229] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:38.883291 27468 raft_consensus.cc:2258] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:38.883419 27469 tablet_replica.cc:330] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: stopping tablet replica
I20260806 22:42:38.883591 27469 raft_consensus.cc:2229] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:38.883654 27469 raft_consensus.cc:2258] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:38.883667 27468 ts_tablet_manager.cc:1905] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:38.883921 27469 ts_tablet_manager.cc:1905] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:38.885145 27468 ts_tablet_manager.cc:1918] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:38.885216 27468 log.cc:1196] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/a365b2f9840f4ddfaf8f8f1a18355dbb
I20260806 22:42:38.885478 27468 ts_tablet_manager.cc:1939] T a365b2f9840f4ddfaf8f8f1a18355dbb P 0b18d85f26ca4753b60c11a7a974a693: Deleting consensus metadata
I20260806 22:42:38.885787 27469 ts_tablet_manager.cc:1918] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:38.885850 27469 log.cc:1196] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/a365b2f9840f4ddfaf8f8f1a18355dbb
I20260806 22:42:38.885836 26667 catalog_manager.cc:4841] TS 0b18d85f26ca4753b60c11a7a974a693 (127.19.74.194:34653): tablet a365b2f9840f4ddfaf8f8f1a18355dbb (table test-table [id=9987c640bb134d4eb5663d837c517764]) successfully deleted
I20260806 22:42:38.886142 27469 ts_tablet_manager.cc:1939] T a365b2f9840f4ddfaf8f8f1a18355dbb P 88a22fda19bd44f68b6c7b93bd2b74c4: Deleting consensus metadata
I20260806 22:42:38.886528 26668 catalog_manager.cc:4841] TS 88a22fda19bd44f68b6c7b93bd2b74c4 (127.19.74.195:43029): tablet a365b2f9840f4ddfaf8f8f1a18355dbb (table test-table [id=9987c640bb134d4eb5663d837c517764]) successfully deleted
I20260806 22:42:38.888933 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.888733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:37361 (local address 127.0.0.1:35886)
0806 22:42:38.888832 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.888857 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:37361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 22:42:38.888999 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 27069
W20260806 22:42:38.889104 26668 catalog_manager.cc:4584] TS 0df6c1f5bac54740a8d25e6118c6cdac (127.19.74.193:37361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a365b2f9840f4ddfaf8f8f1a18355dbb: Network error: Client connection negotiation failed: client connection to 127.19.74.193:37361: connect: Connection refused (error 111)
I20260806 22:42:38.896519 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 27208
I20260806 22:42:38.902117 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 26542
I20260806 22:42:38.907960 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 26738
2026-08-06T22:42:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6437 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:38 dist-test-slave-pgkr krb5kdc[27473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:38 dist-test-slave-pgkr krb5kdc[27473](info): set up 2 sockets
Aug 06 22:42:38 dist-test-slave-pgkr krb5kdc[27473](info): commencing operation
krb5kdc: starting...
W20260806 22:42:39.952997 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.254.keytab.
Entry for principal oryx/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.254.keytab.
I20260806 22:42:40.072988 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37569
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.254.keytab
--principal=oryx/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:37569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:40.154604 27486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:40.154901 27486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:40.154939 27486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:40.156095 27486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:40.156181 27486 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:42:40.156203 27486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:40.156226 27486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:40.156253 27486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:40.156281 27486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:40.157842 27486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:37569
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.254.keytab
--principal=oryx/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:40.158170 27486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:40.158439 27486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:40.163461 27492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:40.163478 27494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:40.163472 27491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:40.163692 27486 server_base.cc:1034] running on GCE node
I20260806 22:42:40.163964 27486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:40.164263 27486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:40.165431 27486 hybrid_clock.cc:648] HybridClock initialized: now 1786056160165418 us; error 46 us; skew 500 ppm
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:40.171995 27486 init.cc:377] Logged in from keytab as oryx/127.19.74.254@KRBTEST.COM (short username oryx)
I20260806 22:42:40.172461 27486 webserver.cc:460] Webserver started at http://127.19.74.254:34757/ using document root <none> and password file <none>
I20260806 22:42:40.172719 27486 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:40.172827 27486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:40.173017 27486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:40.174111 27486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "22e4d688d24142b88a2d7b4f717b92c4"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "90a51b5e6750ece8fb8c924c0dc93468"
server_key_iv: "271e05cdffee79aac4aadc75d09c9ceb"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.174526 27486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "22e4d688d24142b88a2d7b4f717b92c4"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "90a51b5e6750ece8fb8c924c0dc93468"
server_key_iv: "271e05cdffee79aac4aadc75d09c9ceb"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.175899 27486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:40.176641 27501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.176823 27486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:40.176934 27486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "22e4d688d24142b88a2d7b4f717b92c4"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "90a51b5e6750ece8fb8c924c0dc93468"
server_key_iv: "271e05cdffee79aac4aadc75d09c9ceb"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.177008 27486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:40.187814 27486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:40.190095 27486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:40.190266 27486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:40.195145 27486 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:37569
I20260806 22:42:40.195228 27563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:37569 every 8 connection(s)
I20260806 22:42:40.195544 27486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:40.196257 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.197729 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 27486
I20260806 22:42:40.197846 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:40.198035 19755 external_mini_cluster.cc:1428] Setting key ba8f31744d7ac6c2d1a6b86627e31e42
I20260806 22:42:40.198881 27564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: Bootstrap starting.
I20260806 22:42:40.199585 27564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.199978 27564 log.cc:824] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:40.200732 27564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: No bootstrap required, opened a new log
I20260806 22:42:40.202934 27564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e4d688d24142b88a2d7b4f717b92c4" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37569 } }
I20260806 22:42:40.203133 27564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.203187 27564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e4d688d24142b88a2d7b4f717b92c4, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.203320 27564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e4d688d24142b88a2d7b4f717b92c4" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37569 } }
I20260806 22:42:40.203430 27564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:40.203467 27564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:40.203526 27564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:40.204229 27564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e4d688d24142b88a2d7b4f717b92c4" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37569 } }
I20260806 22:42:40.204362 27564 leader_election.cc:302] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22e4d688d24142b88a2d7b4f717b92c4; no voters:
I20260806 22:42:40.204546 27564 leader_election.cc:288] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:40.204619 27569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:40.204826 27564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:40.204849 27569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [term 1 LEADER]: Becoming Leader. State: Replica: 22e4d688d24142b88a2d7b4f717b92c4, State: Running, Role: LEADER
I20260806 22:42:40.204949 27569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e4d688d24142b88a2d7b4f717b92c4" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37569 } }
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:40.205358 27569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22e4d688d24142b88a2d7b4f717b92c4. Latest consensus state: current_term: 1 leader_uuid: "22e4d688d24142b88a2d7b4f717b92c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e4d688d24142b88a2d7b4f717b92c4" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37569 } } }
I20260806 22:42:40.205368 27570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22e4d688d24142b88a2d7b4f717b92c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e4d688d24142b88a2d7b4f717b92c4" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37569 } } }
I20260806 22:42:40.205476 27569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:40.205499 27570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:40.205909 27579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:40.207116 27579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:40.208947 27579 catalog_manager.cc:1347] Generated new cluster ID: 661893f970d2401ba56fd852a777cbcb
I20260806 22:42:40.208992 27579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:40.209491 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.198596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.19.74.254:37569)
0806 22:42:40.198767 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.198775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.198800 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:42:40.199260 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.199267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.199315 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.199430 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.200285 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.201263 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.202323 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.202780 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.202864 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.205840 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.205845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.205846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.205858 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.207945 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.208324 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.208327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.208328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.208429 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.208626 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.208627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.208627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.208902 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.209043 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.209167 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.209305 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260806 22:42:40.223474 27579 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:40.223963 27579 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:40.227206 27579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: Generated new TSK 0
I20260806 22:42:40.227376 27579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab.
Entry for principal oryx/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab.
I20260806 22:42:40.257048 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37569
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab
--principal=oryx/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:40.338102 27591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:40.338290 27591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:40.338312 27591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:40.339488 27591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:40.339529 27591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:40.339548 27591 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:40.339607 27591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:40.341063 27591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab
--principal=oryx/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:37569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:40.341295 27591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:40.341526 27591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:40.346464 27601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:40.346464 27597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:40.346480 27596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:40.346675 27591 server_base.cc:1034] running on GCE node
I20260806 22:42:40.347004 27591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:40.347229 27591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:40.348379 27591 hybrid_clock.cc:648] HybridClock initialized: now 1786056160348370 us; error 26 us; skew 500 ppm
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:40.350818 27591 init.cc:377] Logged in from keytab as oryx/127.19.74.193@KRBTEST.COM (short username oryx)
I20260806 22:42:40.351222 27591 webserver.cc:460] Webserver started at http://127.19.74.193:46501/ using document root <none> and password file <none>
I20260806 22:42:40.351428 27591 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:40.351488 27591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:40.351625 27591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:40.352588 27591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "91f23da657e8463fb5bfc2c487aa2013"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "3a4bf93e8da07214815e513fc1e74233"
server_key_iv: "1286af74d86dfb70b12cce53c72cfc1d"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.352929 27591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "91f23da657e8463fb5bfc2c487aa2013"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "3a4bf93e8da07214815e513fc1e74233"
server_key_iv: "1286af74d86dfb70b12cce53c72cfc1d"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.354211 27591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:42:40.354971 27608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.355139 27591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.355247 27591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "91f23da657e8463fb5bfc2c487aa2013"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "3a4bf93e8da07214815e513fc1e74233"
server_key_iv: "1286af74d86dfb70b12cce53c72cfc1d"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.355319 27591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:40.363736 27591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:40.366149 27591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:40.366283 27591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:40.366843 27591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:40.366928 27591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.366981 27591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:40.367024 27591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:40.373934 27591 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:36195
I20260806 22:42:40.374001 27725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:36195 every 8 connection(s)
I20260806 22:42:40.374343 27591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:40.378206 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.367268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:41359 (local address 127.19.74.254:37569)
0806 22:42:40.367312 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.367314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.367753 (+ 439us) server_negotiation.cc:407] Connection header received
0806 22:42:40.368368 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.368370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.368383 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.368414 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.369248 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.370012 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.371485 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.371788 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.371826 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.374672 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.374674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.374675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.374684 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.376547 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.377024 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.377025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.377026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.377116 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.377402 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.377403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.377403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.377576 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.377665 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.377958 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.378073 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:40.378307 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.367412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37569 (local address 127.19.74.193:41359)
0806 22:42:40.367641 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.367676 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.368249 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.368578 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.368581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.368886 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:40.369139 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.369144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.370138 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.370139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.371382 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.371385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.371921 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.371922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.372070 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.372619 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:40.372629 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:40.374511 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:40.376645 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.376652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.376655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.376946 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.377207 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.377208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.377208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.377337 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.377660 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:40.377666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:40.377814 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.377932 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.378080 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":75,"threads_started":1}
I20260806 22:42:40.378821 27726 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37569
I20260806 22:42:40.378922 27726 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:40.379099 27726 heartbeater.cc:511] Master 127.19.74.254:37569 requested a full tablet report, sending...
I20260806 22:42:40.379667 27518 ts_manager.cc:194] Registered new tserver with Master: 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195)
I20260806 22:42:40.380336 27518 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:41359
I20260806 22:42:40.382000 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 27591
I20260806 22:42:40.382130 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:40.382344 19755 external_mini_cluster.cc:1428] Setting key 1061d314a78a583eab747b15ebcd6819
I20260806 22:42:40.385731 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.379803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37569 (local address 127.19.74.193:36673)
0806 22:42:40.379934 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.379943 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.379991 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.380707 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.380709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.380877 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:40.380926 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.380928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.381584 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.381585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.382068 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.382069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.382478 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.382479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.382543 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.383008 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:40.383013 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:40.383280 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:40.384874 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.384874 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.384875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.385036 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.385239 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.385240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.385240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.385283 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.385497 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:40.385498 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:40.385541 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.385583 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.385661 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":81,"threads_started":1}
I20260806 22:42:40.385804 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.380570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:36673 (local address 127.19.74.254:37569)
0806 22:42:40.380602 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.380604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.380610 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:42:40.380625 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.380626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.380633 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.380687 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.380998 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.381480 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.382187 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.382390 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.382460 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.383383 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.383384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.383384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.383391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.384780 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.385113 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.385114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.385114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.385177 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.385319 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.385320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.385320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.385441 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.385522 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.385612 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.385685 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
WARNING: no policy specified for oryx/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab.
Entry for principal oryx/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab.
I20260806 22:42:40.409114 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37569
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab
--principal=oryx/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:40.492496 27735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:40.492709 27735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:40.492735 27735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:40.493908 27735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:40.493952 27735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:40.493975 27735 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:40.494016 27735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:40.495494 27735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab
--principal=oryx/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:37569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:40.495689 27735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:40.495874 27735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:40.500989 27741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:40.501094 27735 server_base.cc:1034] running on GCE node
W20260806 22:42:40.500998 27740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:40.500998 27743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:40.501540 27735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:40.501801 27735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:40.502955 27735 hybrid_clock.cc:648] HybridClock initialized: now 1786056160502936 us; error 31 us; skew 500 ppm
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:40.505447 27735 init.cc:377] Logged in from keytab as oryx/127.19.74.194@KRBTEST.COM (short username oryx)
I20260806 22:42:40.505820 27735 webserver.cc:460] Webserver started at http://127.19.74.194:46417/ using document root <none> and password file <none>
I20260806 22:42:40.506083 27735 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:40.506134 27735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:40.506222 27735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:40.507269 27735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "1970a3f92238444d9b96617fe1f695c6"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "5f40e13c2986776063a9ec72b6657dcb"
server_key_iv: "69798f9f1d6451ac455a1c74a3091168"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.507567 27735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "1970a3f92238444d9b96617fe1f695c6"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "5f40e13c2986776063a9ec72b6657dcb"
server_key_iv: "69798f9f1d6451ac455a1c74a3091168"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.508803 27735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:40.509440 27750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.509645 27735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:40.509757 27735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "1970a3f92238444d9b96617fe1f695c6"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "5f40e13c2986776063a9ec72b6657dcb"
server_key_iv: "69798f9f1d6451ac455a1c74a3091168"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.509828 27735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:40.522141 27735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:40.523924 27735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:40.524065 27735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:40.524580 27735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:40.524642 27735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.524673 27735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:40.524703 27735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:40.532330 27735 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:34253
I20260806 22:42:40.532356 27866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:34253 every 8 connection(s)
I20260806 22:42:40.532750 27735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:40.534216 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 27735
I20260806 22:42:40.534330 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:40.534544 19755 external_mini_cluster.cc:1428] Setting key 756acb1603ac5d4a4983c6589c4f57e1
I20260806 22:42:40.535492 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.525013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:48527 (local address 127.19.74.254:37569)
0806 22:42:40.525089 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.525091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.525253 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:42:40.525690 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.525691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.525705 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.525788 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.526587 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.527461 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.529066 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.529436 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.529530 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.532487 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.532489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.532489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.532498 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.533938 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.534428 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.534430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.534430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.534487 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.534824 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.534825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.534826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.534988 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.535056 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.535344 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.535425 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:42:40.535789 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.524985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37569 (local address 127.19.74.194:48527)
0806 22:42:40.525121 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.525158 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.525588 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.525833 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.525836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.526184 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:40.526474 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.526480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.527562 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.527563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.528923 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.528926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.529549 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.529550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.529712 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.530254 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:40.530263 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:40.532378 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:40.534037 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.534044 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.534046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.534325 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.534557 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.534557 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.534558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.534730 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.535048 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:40.535055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:40.535192 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.535306 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.535496 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260806 22:42:40.536346 27867 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37569
I20260806 22:42:40.536487 27867 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:40.536746 27867 heartbeater.cc:511] Master 127.19.74.254:37569 requested a full tablet report, sending...
I20260806 22:42:40.537227 27518 ts_manager.cc:194] Registered new tserver with Master: 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253)
I20260806 22:42:40.537770 27518 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:48527
WARNING: no policy specified for oryx/127.19.74.195@KRBTEST.COM; defaulting to no policy
I20260806 22:42:40.544163 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.537474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:38279 (local address 127.19.74.254:37569)
0806 22:42:40.537514 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.537516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.538787 (+ 1271us) server_negotiation.cc:407] Connection header received
0806 22:42:40.538860 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.538860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.538871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.538927 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.539249 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.539715 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.540414 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.540604 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.540681 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.541566 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.541568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.541568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.541574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.543021 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.543377 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.543378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.543378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.543431 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.543632 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.543633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.543634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.543775 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.543851 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.544004 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.544078 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:40.544179 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.537429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37569 (local address 127.19.74.194:38279)
0806 22:42:40.538715 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.538725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.538800 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.538932 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.538933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.539118 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:40.539166 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.539167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.539808 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.539809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.540318 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.540320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.540681 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.540682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.540748 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.541177 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:40.541183 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:40.541457 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:40.543122 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.543123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.543123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.543305 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.543507 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.543508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.543508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.543570 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.543843 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:40.543844 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:40.543889 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.543976 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.544077 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1255,"thread_start_us":79,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab.
Entry for principal oryx/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab.
I20260806 22:42:40.559455 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37569
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab
--principal=oryx/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:40.641654 27876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:40.641866 27876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:40.641889 27876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:40.642983 27876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:40.643023 27876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:40.643043 27876 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:42:40.643060 27876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:40.644591 27876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab
--principal=oryx/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.74.254:37569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:40.644789 27876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:40.644959 27876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:40.649842 27882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:40.649852 27884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:40.649995 27876 server_base.cc:1034] running on GCE node
W20260806 22:42:40.649852 27881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:40.650276 27876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:40.650573 27876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:40.651749 27876 hybrid_clock.cc:648] HybridClock initialized: now 1786056160651728 us; error 36 us; skew 500 ppm
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:40.654065 27876 init.cc:377] Logged in from keytab as oryx/127.19.74.195@KRBTEST.COM (short username oryx)
I20260806 22:42:40.654467 27876 webserver.cc:460] Webserver started at http://127.19.74.195:37789/ using document root <none> and password file <none>
I20260806 22:42:40.654739 27876 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:40.654805 27876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:40.654938 27876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:40.655982 27876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "3ad4dc547d9741a9a590872560be60ef"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "17221dbc08d036759b651ee6fae58605"
server_key_iv: "287faa20694b7d7d2bb0e2aaea05ef49"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.656347 27876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "3ad4dc547d9741a9a590872560be60ef"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "17221dbc08d036759b651ee6fae58605"
server_key_iv: "287faa20694b7d7d2bb0e2aaea05ef49"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.657723 27876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:40.658510 27891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.658694 27876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.658797 27876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "3ad4dc547d9741a9a590872560be60ef"
format_stamp: "Formatted at 2026-08-06 22:42:40 on dist-test-slave-pgkr"
server_key: "17221dbc08d036759b651ee6fae58605"
server_key_iv: "287faa20694b7d7d2bb0e2aaea05ef49"
server_key_version: "encryptionkey@0"
I20260806 22:42:40.658872 27876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:40.681170 27876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:40.683588 27876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:40.683758 27876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:40.684288 27876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:40.684347 27876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.684387 27876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:40.684417 27876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.690935 27876 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:37493
I20260806 22:42:40.690966 28008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:37493 every 8 connection(s)
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:40.691431 27876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:40.694416 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.684784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:33109 (local address 127.19.74.254:37569)
0806 22:42:40.684853 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.684856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.685048 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:42:40.685517 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.685519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.685533 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.685617 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.686443 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.687317 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.688871 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.689186 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.689272 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.691912 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.691913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.691913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.691919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.693320 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.693689 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.693689 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.693690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.693738 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.693938 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.693938 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.693938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.694030 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.694079 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.694296 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.694354 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:42:40.694636 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.684764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37569 (local address 127.19.74.195:33109)
0806 22:42:40.684908 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.684966 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.685428 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.685705 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.685710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.686050 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:40.686340 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.686348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.687428 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.687429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.688795 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.688798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.689287 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.689288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.689386 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.689927 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:40.689939 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:40.691798 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:40.693394 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.693400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.693401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.693612 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.693794 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.693795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.693795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.693886 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.694082 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:40.694086 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:40.694192 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.694270 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.694382 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260806 22:42:40.695182 28009 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37569
I20260806 22:42:40.695247 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 27876
I20260806 22:42:40.695278 28009 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:40.695370 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:40.695473 28009 heartbeater.cc:511] Master 127.19.74.254:37569 requested a full tablet report, sending...
I20260806 22:42:40.695582 19755 external_mini_cluster.cc:1428] Setting key 3d08379622fa1c5fb14f34ccd0cfac2f
I20260806 22:42:40.695881 27518 ts_manager.cc:194] Registered new tserver with Master: 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:40.696352 27518 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:33109
I20260806 22:42:40.696691 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:40.703087 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.696659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:44773 (local address 127.19.74.254:37569)
0806 22:42:40.696690 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.696692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.697569 (+ 877us) server_negotiation.cc:407] Connection header received
0806 22:42:40.697634 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.697635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.697645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.697701 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.698023 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.698469 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.699218 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.699422 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.699530 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.700428 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.700429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.700429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.700437 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.702023 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.702321 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.702322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.702322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.702384 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.702564 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.702564 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.702565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.702742 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.702813 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.702980 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.703035 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 22:42:40.703222 28015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.696633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37569 (local address 127.19.74.195:44773)
0806 22:42:40.697493 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.697503 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.697582 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.697706 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.697707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.697880 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:40.697941 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.697942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.698575 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.698576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.699133 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.699135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.699533 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.699534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.699606 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.700045 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:40.700049 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:40.700341 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:40.702099 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.702100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.702101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.702252 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.702456 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.702456 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.702456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.702505 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.702822 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:40.702824 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:40.702873 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.702961 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.703069 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":88,"threads_started":1}
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:42:40.719736 27567 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:40.714843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.19.74.254:37569)
0806 22:42:40.714899 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.714900 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.714936 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:42:40.714953 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.714954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.714962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.715000 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.715512 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.715979 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.716657 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.716826 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.716875 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.719575 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51998: BlockingRecv error: recv got EOF from 127.0.0.1:51998 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:40.726302 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.720460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.19.74.254:37569)
0806 22:42:40.720532 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.720534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.720618 (+ 84us) server_negotiation.cc:407] Connection header received
0806 22:42:40.720668 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.720669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.720676 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.720721 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.721131 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.721535 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.722118 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.722274 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.722320 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.724282 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.724283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.724283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.724289 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.725350 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.725681 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.725681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.725682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.725727 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.725915 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.725915 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.725916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.726000 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.726068 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.726163 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.726214 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:42:40.727988 27517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:40.729511 27517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:40.735419 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.731668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:34253 (local address 127.0.0.1:53986)
0806 22:42:40.731827 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.731866 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.731936 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.732191 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.732194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.732197 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:40.732269 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.732271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.733284 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.733285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.734220 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.734223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.735099 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.735100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.735185 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.735199 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.735284 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.735350 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":47,"threads_started":1}
I20260806 22:42:40.735466 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.731782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53986 (local address 127.19.74.194:34253)
0806 22:42:40.732024 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.732029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.732038 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:40.732081 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.732083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.732103 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.732187 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:40.732368 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.733194 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.734330 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.735001 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.735090 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.735161 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.735308 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.735364 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
I20260806 22:42:40.736212 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.731694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36195 (local address 127.0.0.1:59982)
0806 22:42:40.731951 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.731962 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.732013 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.732528 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.732529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.732531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:40.732580 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.732581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.733901 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.733902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.734895 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.734897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.735958 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.735959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.736019 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.736023 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.736082 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.736125 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":72,"threads_started":1}
I20260806 22:42:40.736236 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.731813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.19.74.193:36195)
0806 22:42:40.732349 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.732355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.732365 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:40.732406 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.732408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.732423 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.732653 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:40.732712 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.733814 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.734973 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.735804 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.735934 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.736009 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.736105 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.736142 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":84,"threads_started":1}
I20260806 22:42:40.736486 27801 tablet_service.cc:1467] Processing CreateTablet for tablet 1293ea569588450d9f4ddf5362450f2c (DEFAULT_TABLE table=test-table [id=d72f02a54c5c4384b17bfb9000b35476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:40.736902 27801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1293ea569588450d9f4ddf5362450f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.737118 27658 tablet_service.cc:1467] Processing CreateTablet for tablet 1293ea569588450d9f4ddf5362450f2c (DEFAULT_TABLE table=test-table [id=d72f02a54c5c4384b17bfb9000b35476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:40.737377 28035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.731869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:37493 (local address 127.0.0.1:59040)
0806 22:42:40.732088 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.732098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.732135 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.732636 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.732637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.732639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:40.732668 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.732669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.734966 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.734966 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.736119 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.736122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.737151 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.737152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.737196 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.737199 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.737269 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.737312 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":54,"threads_started":1}
I20260806 22:42:40.737475 27658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1293ea569588450d9f4ddf5362450f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.737707 28036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.732047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59040 (local address 127.19.74.195:37493)
0806 22:42:40.732430 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.732434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.732443 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:40.732477 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.732479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.732494 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.732564 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:40.732744 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.733815 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.736207 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.736988 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.737053 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.737120 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.737347 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.737385 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":97,"threads_started":1}
I20260806 22:42:40.738210 27942 tablet_service.cc:1467] Processing CreateTablet for tablet 1293ea569588450d9f4ddf5362450f2c (DEFAULT_TABLE table=test-table [id=d72f02a54c5c4384b17bfb9000b35476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:40.738521 27942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1293ea569588450d9f4ddf5362450f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.740990 28037 tablet_bootstrap.cc:492] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Bootstrap starting.
I20260806 22:42:40.741314 28039 tablet_bootstrap.cc:492] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Bootstrap starting.
I20260806 22:42:40.741729 28037 tablet_bootstrap.cc:654] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.741838 28038 tablet_bootstrap.cc:492] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: Bootstrap starting.
I20260806 22:42:40.742003 28039 tablet_bootstrap.cc:654] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.742090 28037 log.cc:824] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:40.742336 28039 log.cc:824] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:40.742491 28038 tablet_bootstrap.cc:654] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.742801 28038 log.cc:824] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:40.742923 28037 tablet_bootstrap.cc:492] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: No bootstrap required, opened a new log
I20260806 22:42:40.742988 28037 ts_tablet_manager.cc:1397] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:40.743217 28039 tablet_bootstrap.cc:492] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: No bootstrap required, opened a new log
I20260806 22:42:40.743297 28039 ts_tablet_manager.cc:1397] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:40.744451 28037 raft_consensus.cc:348] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.744618 28037 raft_consensus.cc:374] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.744673 28037 raft_consensus.cc:729] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1970a3f92238444d9b96617fe1f695c6, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.744791 28037 consensus_queue.cc:260] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.744932 28039 raft_consensus.cc:348] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.745078 28037 ts_tablet_manager.cc:1428] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:42:40.745108 27867 heartbeater.cc:503] Master 127.19.74.254:37569 was elected leader, sending a full tablet report...
I20260806 22:42:40.745103 28039 raft_consensus.cc:374] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.745148 28039 raft_consensus.cc:729] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f23da657e8463fb5bfc2c487aa2013, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.745270 28039 consensus_queue.cc:260] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.745518 28039 ts_tablet_manager.cc:1428] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:40.745579 27726 heartbeater.cc:503] Master 127.19.74.254:37569 was elected leader, sending a full tablet report...
I20260806 22:42:40.746168 28038 tablet_bootstrap.cc:492] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: No bootstrap required, opened a new log
I20260806 22:42:40.746238 28038 ts_tablet_manager.cc:1397] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 22:42:40.748307 28038 raft_consensus.cc:348] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.748440 28038 raft_consensus.cc:374] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.748461 28038 raft_consensus.cc:729] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ad4dc547d9741a9a590872560be60ef, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.748590 28038 consensus_queue.cc:260] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.748869 28038 ts_tablet_manager.cc:1428] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:42:40.748903 28009 heartbeater.cc:503] Master 127.19.74.254:37569 was elected leader, sending a full tablet report...
W20260806 22:42:40.783414 27868 tablet.cc:2369] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:40.862758 28043 raft_consensus.cc:482] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:40.862879 28043 raft_consensus.cc:504] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.863180 28043 leader_election.cc:288] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:40.865909 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.863489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:34099 (local address 127.19.74.193:36195)
0806 22:42:40.863530 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.863533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.863544 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:40.863581 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.863582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.863591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.863670 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:40.863840 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.864289 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.865210 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.865548 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.865623 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.865632 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.865810 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.865843 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:40.865922 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.863412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36195 (local address 127.19.74.194:34099)
0806 22:42:40.863473 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.863485 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.863530 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.863679 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.863681 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.863683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:40.863748 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.863750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.864386 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.864387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.865110 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.865112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.865646 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.865647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.865743 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.865746 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.865804 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.865843 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:42:40.866083 28036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.863644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:39841 (local address 127.19.74.195:37493)
0806 22:42:40.863687 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.863689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.863734 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:42:40.863804 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.863804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.863814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.863877 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:40.864018 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.864466 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.865348 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.865704 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.865778 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.865785 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.865938 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.865992 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:42:40.866111 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.863523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:37493 (local address 127.19.74.194:39841)
0806 22:42:40.863681 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.863690 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.863740 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.863880 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.863881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.863883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:40.863947 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.863949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.864549 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.864550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.865254 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.865256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.865791 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.865792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.865868 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.865871 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.865930 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.865984 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260806 22:42:40.866307 27679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1293ea569588450d9f4ddf5362450f2c" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f23da657e8463fb5bfc2c487aa2013" is_pre_election: true
I20260806 22:42:40.866482 27679 raft_consensus.cc:2454] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 0.
I20260806 22:42:40.866683 27962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1293ea569588450d9f4ddf5362450f2c" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef" is_pre_election: true
I20260806 22:42:40.866791 27754 leader_election.cc:302] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1970a3f92238444d9b96617fe1f695c6, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:40.866830 27962 raft_consensus.cc:2454] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 0.
I20260806 22:42:40.866912 28043 raft_consensus.cc:2781] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:40.866984 28043 raft_consensus.cc:482] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:40.867018 28043 raft_consensus.cc:3037] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:40.868068 28043 raft_consensus.cc:504] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.868249 28043 leader_election.cc:288] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 election: Requested vote from peers 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:40.868419 27679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1293ea569588450d9f4ddf5362450f2c" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f23da657e8463fb5bfc2c487aa2013"
I20260806 22:42:40.868438 27962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1293ea569588450d9f4ddf5362450f2c" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef"
I20260806 22:42:40.868510 27962 raft_consensus.cc:3037] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:40.868501 27679 raft_consensus.cc:3037] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:40.869415 27679 raft_consensus.cc:2454] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 1.
I20260806 22:42:40.869415 27962 raft_consensus.cc:2454] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 1.
I20260806 22:42:40.869582 27754 leader_election.cc:302] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1970a3f92238444d9b96617fe1f695c6, 3ad4dc547d9741a9a590872560be60ef; no voters:
I20260806 22:42:40.869671 28043 raft_consensus.cc:2781] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:40.869823 28043 raft_consensus.cc:686] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 1 LEADER]: Becoming Leader. State: Replica: 1970a3f92238444d9b96617fe1f695c6, State: Running, Role: LEADER
I20260806 22:42:40.869923 28043 consensus_queue.cc:237] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.870847 27517 catalog_manager.cc:5495] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194). New cstate: current_term: 1 leader_uuid: "1970a3f92238444d9b96617fe1f695c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } health_report { overall_health: UNKNOWN } } }
W20260806 22:42:40.875360 27727 tablet.cc:2369] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:40.883965 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.881732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53998 (local address 127.19.74.194:34253)
0806 22:42:40.881795 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.881798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.881807 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:40.881856 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.881857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.881876 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.881952 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:40.882079 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.882505 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.883251 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.883447 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.883529 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.883687 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:40.883777 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:40.883860 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.883881 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.883899 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:42:40.885561 27679 raft_consensus.cc:1261] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Refusing update from remote peer 1970a3f92238444d9b96617fe1f695c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:40.885561 27962 raft_consensus.cc:1261] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Refusing update from remote peer 1970a3f92238444d9b96617fe1f695c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:40.885870 28043 consensus_queue.cc:1035] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:40.886018 28047 consensus_queue.cc:1035] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:40.888298 28050 mvcc.cc:205] Tried to move back new op lower bound from 7315686034984685568 to 7315686034923851776. Current Snapshot: MvccSnapshot[applied={T|T < 7315686034984685568}]
I20260806 22:42:40.888922 28051 mvcc.cc:205] Tried to move back new op lower bound from 7315686034984685568 to 7315686034923851776. Current Snapshot: MvccSnapshot[applied={T|T < 7315686034984685568}]
I20260806 22:42:40.889210 28052 mvcc.cc:205] Tried to move back new op lower bound from 7315686034984685568 to 7315686034923851776. Current Snapshot: MvccSnapshot[applied={T|T < 7315686034984685568 or (T in {7315686034984685568})}]
I20260806 22:42:40.891009 27517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:40.891111 27517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:40.891821 27517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: Sending DeleteTablet for 3 replicas of tablet 1293ea569588450d9f4ddf5362450f2c
I20260806 22:42:40.892150 27801 tablet_service.cc:1514] Processing DeleteTablet for tablet 1293ea569588450d9f4ddf5362450f2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:40 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:53986
I20260806 22:42:40.892252 27658 tablet_service.cc:1514] Processing DeleteTablet for tablet 1293ea569588450d9f4ddf5362450f2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:40 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59982
I20260806 22:42:40.892441 27942 tablet_service.cc:1514] Processing DeleteTablet for tablet 1293ea569588450d9f4ddf5362450f2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:40 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59040
I20260806 22:42:40.892422 27517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:40.892565 27517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:40.892650 28062 tablet_replica.cc:330] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: stopping tablet replica
I20260806 22:42:40.892742 28062 raft_consensus.cc:2229] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:40.892843 28062 raft_consensus.cc:2258] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:40.893013 28063 tablet_replica.cc:330] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: stopping tablet replica
I20260806 22:42:40.893096 28063 raft_consensus.cc:2229] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:40.893432 28062 ts_tablet_manager.cc:1905] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:40.893484 28063 raft_consensus.cc:2258] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:40.893651 28061 tablet_replica.cc:330] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: stopping tablet replica
I20260806 22:42:40.893746 28061 raft_consensus.cc:2229] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:40.893855 28061 raft_consensus.cc:2258] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:40.894147 27801 tablet_service.cc:1467] Processing CreateTablet for tablet f7ef169792a64555bd88a3eb4712e926 (DEFAULT_TABLE table=test-table [id=5fd74bbb49ed49f391b42c4f4de11553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:40.894158 27658 tablet_service.cc:1467] Processing CreateTablet for tablet f7ef169792a64555bd88a3eb4712e926 (DEFAULT_TABLE table=test-table [id=5fd74bbb49ed49f391b42c4f4de11553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:40.894258 27801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ef169792a64555bd88a3eb4712e926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.894258 27658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ef169792a64555bd88a3eb4712e926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.894536 28063 ts_tablet_manager.cc:1905] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:40.894711 27942 tablet_service.cc:1467] Processing CreateTablet for tablet f7ef169792a64555bd88a3eb4712e926 (DEFAULT_TABLE table=test-table [id=5fd74bbb49ed49f391b42c4f4de11553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:40.894829 27942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ef169792a64555bd88a3eb4712e926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.895673 28037 tablet_bootstrap.cc:492] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: Bootstrap starting.
I20260806 22:42:40.896093 28037 tablet_bootstrap.cc:654] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.896250 28038 tablet_bootstrap.cc:492] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: Bootstrap starting.
I20260806 22:42:40.896538 28063 ts_tablet_manager.cc:1918] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:40.896668 28063 log.cc:1196] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/1293ea569588450d9f4ddf5362450f2c
I20260806 22:42:40.897022 28063 ts_tablet_manager.cc:1939] T 1293ea569588450d9f4ddf5362450f2c P 3ad4dc547d9741a9a590872560be60ef: Deleting consensus metadata
I20260806 22:42:40.897984 28039 tablet_bootstrap.cc:492] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: Bootstrap starting.
I20260806 22:42:40.898094 27505 catalog_manager.cc:4841] TS 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493): tablet 1293ea569588450d9f4ddf5362450f2c (table test-table [id=d72f02a54c5c4384b17bfb9000b35476]) successfully deleted
I20260806 22:42:40.896693 28038 tablet_bootstrap.cc:654] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.898365 28039 tablet_bootstrap.cc:654] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.898408 28037 tablet_bootstrap.cc:492] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: No bootstrap required, opened a new log
I20260806 22:42:40.898476 28037 ts_tablet_manager.cc:1397] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 22:42:40.898603 28062 ts_tablet_manager.cc:1918] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:40.898700 28062 log.cc:1196] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/1293ea569588450d9f4ddf5362450f2c
I20260806 22:42:40.898660 28037 raft_consensus.cc:348] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.898739 28037 raft_consensus.cc:374] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.898772 28037 raft_consensus.cc:729] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1970a3f92238444d9b96617fe1f695c6, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.898819 28037 consensus_queue.cc:260] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.898926 28037 ts_tablet_manager.cc:1428] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.898967 28062 ts_tablet_manager.cc:1939] T 1293ea569588450d9f4ddf5362450f2c P 91f23da657e8463fb5bfc2c487aa2013: Deleting consensus metadata
I20260806 22:42:40.899343 27505 catalog_manager.cc:4841] TS 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195): tablet 1293ea569588450d9f4ddf5362450f2c (table test-table [id=d72f02a54c5c4384b17bfb9000b35476]) successfully deleted
I20260806 22:42:40.899680 28061 ts_tablet_manager.cc:1905] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:40.900790 28061 ts_tablet_manager.cc:1918] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:40.900851 28061 log.cc:1196] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/1293ea569588450d9f4ddf5362450f2c
I20260806 22:42:40.901094 28061 ts_tablet_manager.cc:1939] T 1293ea569588450d9f4ddf5362450f2c P 1970a3f92238444d9b96617fe1f695c6: Deleting consensus metadata
I20260806 22:42:40.901569 28038 tablet_bootstrap.cc:492] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: No bootstrap required, opened a new log
I20260806 22:42:40.901638 28038 ts_tablet_manager.cc:1397] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260806 22:42:40.901678 27502 catalog_manager.cc:4841] TS 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253): tablet 1293ea569588450d9f4ddf5362450f2c (table test-table [id=d72f02a54c5c4384b17bfb9000b35476]) successfully deleted
I20260806 22:42:40.901785 28038 raft_consensus.cc:348] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.901834 28038 raft_consensus.cc:374] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.901847 28038 raft_consensus.cc:729] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ad4dc547d9741a9a590872560be60ef, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.901921 28038 consensus_queue.cc:260] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.902015 28038 ts_tablet_manager.cc:1428] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.902518 28039 tablet_bootstrap.cc:492] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: No bootstrap required, opened a new log
I20260806 22:42:40.902567 28039 ts_tablet_manager.cc:1397] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260806 22:42:40.902953 28039 raft_consensus.cc:348] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.903038 28039 raft_consensus.cc:374] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.903060 28039 raft_consensus.cc:729] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f23da657e8463fb5bfc2c487aa2013, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.903115 28039 consensus_queue.cc:260] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.903235 28039 ts_tablet_manager.cc:1428] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:40.912174 28055 raft_consensus.cc:482] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:40.912304 28055 raft_consensus.cc:504] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.912442 28055 leader_election.cc:288] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:40.912679 27962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ef169792a64555bd88a3eb4712e926" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef" is_pre_election: true
I20260806 22:42:40.912669 27679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ef169792a64555bd88a3eb4712e926" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f23da657e8463fb5bfc2c487aa2013" is_pre_election: true
I20260806 22:42:40.912768 27679 raft_consensus.cc:2454] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 0.
I20260806 22:42:40.912770 27962 raft_consensus.cc:2454] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 0.
I20260806 22:42:40.912918 27754 leader_election.cc:302] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1970a3f92238444d9b96617fe1f695c6, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:40.913094 28055 raft_consensus.cc:2781] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:40.913148 28055 raft_consensus.cc:482] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:40.913173 28055 raft_consensus.cc:3037] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:40.913949 28055 raft_consensus.cc:504] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.914078 28055 leader_election.cc:288] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 election: Requested vote from peers 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:40.914245 27962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ef169792a64555bd88a3eb4712e926" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef"
I20260806 22:42:40.914244 27679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ef169792a64555bd88a3eb4712e926" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f23da657e8463fb5bfc2c487aa2013"
I20260806 22:42:40.914316 27962 raft_consensus.cc:3037] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:40.914335 27679 raft_consensus.cc:3037] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:40.915050 27679 raft_consensus.cc:2454] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 1.
I20260806 22:42:40.915119 27962 raft_consensus.cc:2454] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 1.
I20260806 22:42:40.915220 27754 leader_election.cc:302] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1970a3f92238444d9b96617fe1f695c6, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:40.915369 28055 raft_consensus.cc:2781] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:40.915436 28055 raft_consensus.cc:686] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 1 LEADER]: Becoming Leader. State: Replica: 1970a3f92238444d9b96617fe1f695c6, State: Running, Role: LEADER
I20260806 22:42:40.915474 28055 consensus_queue.cc:237] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.916105 27518 catalog_manager.cc:5495] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194). New cstate: current_term: 1 leader_uuid: "1970a3f92238444d9b96617fe1f695c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } health_report { overall_health: UNKNOWN } } }
W20260806 22:42:40.941995 28010 tablet.cc:2369] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:42:40 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056160, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.254@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:40.963407 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.955556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52008 (local address 127.19.74.254:37569)
0806 22:42:40.955626 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:40.955629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:40.955709 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:42:40.955757 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:40.955758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:40.955770 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:40.955849 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:40.956294 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.957059 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.957850 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.958161 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.958238 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.960757 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:40.960758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:40.960758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:40.960764 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:40.962184 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.962695 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.962696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.962697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.962751 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:40.962958 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:40.962959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:40.962959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:40.963063 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:40.963116 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:40.963284 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:42:40.963347 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:42:40.963515 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.955453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37569 (local address 127.0.0.1:52008)
0806 22:42:40.955639 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.955648 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:40.955708 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:40.955862 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:40.955863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:40.956161 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:40.956211 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.956213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.957144 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.957144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.957765 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:40.957768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:40.958236 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:40.958237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:40.958279 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:40.958945 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:40.958950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:40.960676 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:40.962294 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.962302 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.962304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.962590 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.962818 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:40.962819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:40.962819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:40.962894 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:40.963128 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:40.963135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:40.963186 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:42:40.963262 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:42:40.963421 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":45,"thread_start_us":86,"threads_started":1}
I20260806 22:42:40.965512 27518 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:52008:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:42:40.965732 27518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:40.967391 27942 tablet_service.cc:1467] Processing CreateTablet for tablet 293b1518f2104cd3af5dfa01ed5f5bff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76ea621089f44205bba8c8a7a8df0a1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:40.967367 27658 tablet_service.cc:1467] Processing CreateTablet for tablet 293b1518f2104cd3af5dfa01ed5f5bff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76ea621089f44205bba8c8a7a8df0a1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:40.967538 27942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293b1518f2104cd3af5dfa01ed5f5bff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.967540 27658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293b1518f2104cd3af5dfa01ed5f5bff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.969069 28038 tablet_bootstrap.cc:492] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: Bootstrap starting.
I20260806 22:42:40.969092 28039 tablet_bootstrap.cc:492] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: Bootstrap starting.
I20260806 22:42:40.969163 27801 tablet_service.cc:1467] Processing CreateTablet for tablet 293b1518f2104cd3af5dfa01ed5f5bff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76ea621089f44205bba8c8a7a8df0a1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:42:40.969333 27801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293b1518f2104cd3af5dfa01ed5f5bff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:40.969503 28038 tablet_bootstrap.cc:654] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.969559 28039 tablet_bootstrap.cc:654] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.970348 28038 tablet_bootstrap.cc:492] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: No bootstrap required, opened a new log
I20260806 22:42:40.970402 28038 ts_tablet_manager.cc:1397] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:40.970546 28038 raft_consensus.cc:348] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.970674 28038 raft_consensus.cc:374] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.970711 28038 raft_consensus.cc:729] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ad4dc547d9741a9a590872560be60ef, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.970750 28038 consensus_queue.cc:260] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.970850 28038 ts_tablet_manager.cc:1428] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:40.970912 28037 tablet_bootstrap.cc:492] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: Bootstrap starting.
I20260806 22:42:40.970993 28045 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:40.971105 28045 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:40.971308 28037 tablet_bootstrap.cc:654] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:40.972021 28037 tablet_bootstrap.cc:492] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: No bootstrap required, opened a new log
I20260806 22:42:40.972074 28037 ts_tablet_manager.cc:1397] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:40.972371 28037 raft_consensus.cc:348] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.972427 28037 raft_consensus.cc:374] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.972441 28037 raft_consensus.cc:729] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1970a3f92238444d9b96617fe1f695c6, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.972493 28037 consensus_queue.cc:260] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.972587 28037 ts_tablet_manager.cc:1428] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.972689 28055 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:40.972813 28055 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:40.973168 28039 tablet_bootstrap.cc:492] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: No bootstrap required, opened a new log
I20260806 22:42:40.973227 28039 ts_tablet_manager.cc:1397] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 22:42:40.973373 28039 raft_consensus.cc:348] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.973459 28039 raft_consensus.cc:374] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:40.973480 28039 raft_consensus.cc:729] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f23da657e8463fb5bfc2c487aa2013, State: Initialized, Role: FOLLOWER
I20260806 22:42:40.973521 28039 consensus_queue.cc:260] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:40.973659 28039 ts_tablet_manager.cc:1428] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:40.973762 28044 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:40.973867 28044 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:41.352723 28044 raft_consensus.cc:482] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:41.352851 28044 raft_consensus.cc:504] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.353273 28044 leader_election.cc:288] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:41.356496 28036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.353575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:55941 (local address 127.19.74.195:37493)
0806 22:42:41.353674 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.353676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.353782 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:42:41.353877 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.353878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.353887 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.353958 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:41.354153 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.354708 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.355770 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.356129 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.356217 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.356226 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.356383 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.356419 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:42:41.356499 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.353548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:52995 (local address 127.19.74.194:34253)
0806 22:42:41.353601 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.353603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.353769 (+ 166us) server_negotiation.cc:407] Connection header received
0806 22:42:41.353862 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.353863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.353882 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.353942 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:41.354098 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.354698 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.355766 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.356139 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.356234 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.356242 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.356392 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.356431 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:42:41.356539 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.353465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:34253 (local address 127.19.74.193:52995)
0806 22:42:41.353683 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.353693 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:41.353780 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:41.353946 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:41.353946 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:41.353948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:41.354027 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.354029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.354789 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.354790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.355671 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.355674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.356235 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.356236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.356308 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.356312 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:41.356373 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:42:41.356416 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":29,"thread_start_us":125,"threads_started":1}
I20260806 22:42:41.356545 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.353485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:37493 (local address 127.19.74.193:55941)
0806 22:42:41.353694 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.353705 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:41.353808 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:41.353964 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:41.353965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:41.353966 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:41.354044 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.354045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.354793 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.354794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.355672 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.355675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.356217 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.356218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.356301 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.356305 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:41.356375 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:42:41.356418 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":37,"thread_start_us":126,"threads_started":1}
I20260806 22:42:41.356875 27821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293b1518f2104cd3af5dfa01ed5f5bff" candidate_uuid: "91f23da657e8463fb5bfc2c487aa2013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1970a3f92238444d9b96617fe1f695c6" is_pre_election: true
I20260806 22:42:41.356989 27962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293b1518f2104cd3af5dfa01ed5f5bff" candidate_uuid: "91f23da657e8463fb5bfc2c487aa2013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef" is_pre_election: true
I20260806 22:42:41.357052 27821 raft_consensus.cc:2454] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f23da657e8463fb5bfc2c487aa2013 in term 0.
I20260806 22:42:41.357100 27962 raft_consensus.cc:2454] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f23da657e8463fb5bfc2c487aa2013 in term 0.
I20260806 22:42:41.357318 27609 leader_election.cc:302] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1970a3f92238444d9b96617fe1f695c6, 3ad4dc547d9741a9a590872560be60ef, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:41.357462 28044 raft_consensus.cc:2781] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:41.357518 28044 raft_consensus.cc:482] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:41.357535 28044 raft_consensus.cc:3037] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.358343 28044 raft_consensus.cc:504] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.358491 28044 leader_election.cc:288] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [CANDIDATE]: Term 1 election: Requested vote from peers 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:41.358670 27821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293b1518f2104cd3af5dfa01ed5f5bff" candidate_uuid: "91f23da657e8463fb5bfc2c487aa2013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1970a3f92238444d9b96617fe1f695c6"
I20260806 22:42:41.358706 27962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293b1518f2104cd3af5dfa01ed5f5bff" candidate_uuid: "91f23da657e8463fb5bfc2c487aa2013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef"
I20260806 22:42:41.358745 27821 raft_consensus.cc:3037] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.358767 27962 raft_consensus.cc:3037] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.358753 28055 consensus_queue.cc:1035] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:41.359503 27962 raft_consensus.cc:2454] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f23da657e8463fb5bfc2c487aa2013 in term 1.
I20260806 22:42:41.359608 27821 raft_consensus.cc:2454] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f23da657e8463fb5bfc2c487aa2013 in term 1.
I20260806 22:42:41.359695 27612 leader_election.cc:302] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ad4dc547d9741a9a590872560be60ef, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:41.359786 28044 raft_consensus.cc:2781] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:41.359952 28044 raft_consensus.cc:686] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [term 1 LEADER]: Becoming Leader. State: Replica: 91f23da657e8463fb5bfc2c487aa2013, State: Running, Role: LEADER
I20260806 22:42:41.360042 28044 consensus_queue.cc:237] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.360877 27517 catalog_manager.cc:5495] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:41.360984 28055 consensus_queue.cc:1035] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:41.361845 28081 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: TxnStatusTablet state changed. Reason: New leader 91f23da657e8463fb5bfc2c487aa2013. Latest consensus state: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:41.361948 28081 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:41.362123 28084 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:42:41.362258 28084 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:42:41.414688 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.411929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36195 (local address 127.0.0.1:59994)
0806 22:42:41.412039 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.412052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:41.412116 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:41.412383 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:41.412384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:41.412386 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:41.412443 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.412445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.413089 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.413090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.413726 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.413728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.414111 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.414112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.414183 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.414198 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.414483 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.414494 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:42:41.414549 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:42:41.414599 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:42:41.414702 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.411995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.19.74.193:36195)
0806 22:42:41.412217 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.412219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.412225 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:42:41.412267 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.412267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.412285 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.412363 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:41.412524 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.412997 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.413816 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.414034 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.414111 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.414307 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.414409 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.414487 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.414552 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.414575 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":63,"threads_started":1}
I20260806 22:42:41.415859 27961 raft_consensus.cc:1261] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Refusing update from remote peer 91f23da657e8463fb5bfc2c487aa2013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:41.415951 27821 raft_consensus.cc:1261] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Refusing update from remote peer 91f23da657e8463fb5bfc2c487aa2013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:41.416108 28044 consensus_queue.cc:1035] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:41.416280 28044 consensus_queue.cc:1035] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:41.416828 28043 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: TxnStatusTablet state changed. Reason: New leader 91f23da657e8463fb5bfc2c487aa2013. Latest consensus state: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:41.416925 28043 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:41.417363 28045 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: TxnStatusTablet state changed. Reason: New leader 91f23da657e8463fb5bfc2c487aa2013. Latest consensus state: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:41.417455 28045 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:41.417608 28081 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:41.417686 28081 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:41.417861 28043 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:41.417944 28043 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 1970a3f92238444d9b96617fe1f695c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:41.418294 28081 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:41.418367 28081 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 91f23da657e8463fb5bfc2c487aa2013: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:42:41.418932 28045 tablet_replica.cc:439] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } } }
I20260806 22:42:41.419008 28045 tablet_replica.cc:442] T 293b1518f2104cd3af5dfa01ed5f5bff P 3ad4dc547d9741a9a590872560be60ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:42:41.424571 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.422074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:36335 (local address 127.19.74.193:36195)
0806 22:42:41.422113 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.422115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.422249 (+ 134us) server_negotiation.cc:407] Connection header received
0806 22:42:41.422314 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.422315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.422326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.422381 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:41.422509 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.423023 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.423828 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.424031 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.424114 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.424333 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.424370 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.424432 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.424493 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.424512 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:41.424635 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.422007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36195 (local address 127.19.74.194:36335)
0806 22:42:41.422182 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.422191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:41.422265 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:41.422383 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:41.422384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:41.422386 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:41.422434 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.422439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.423102 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.423103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.423726 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.423728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.424118 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.424120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.424222 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.424236 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.424433 (+ 197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.424436 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:41.424489 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:42:41.424548 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260806 22:42:41.434176 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.431712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:39393 (local address 127.19.74.194:34253)
0806 22:42:41.431759 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.431761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.431769 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:41.431828 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.431829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.431838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.431883 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:41.432019 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.432455 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.433519 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.433714 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.433767 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.433979 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.434006 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.434049 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.434110 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.434128 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:42:41.434260 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.431516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:34253 (local address 127.19.74.193:39393)
0806 22:42:41.431685 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.431698 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:41.431755 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:41.431890 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:41.431890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:41.431891 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:41.431954 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.431956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.432540 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.432542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.433430 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.433434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.433800 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.433802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.433875 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.433890 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.434054 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.434056 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:41.434109 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:42:41.434152 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260806 22:42:41.442649 27517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:41.442766 27517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:41.443471 27517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: Sending DeleteTablet for 3 replicas of tablet f7ef169792a64555bd88a3eb4712e926
I20260806 22:42:41.443720 27801 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ef169792a64555bd88a3eb4712e926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:53986
I20260806 22:42:41.443804 27942 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ef169792a64555bd88a3eb4712e926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59040
I20260806 22:42:41.443951 27658 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ef169792a64555bd88a3eb4712e926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59982
I20260806 22:42:41.444126 28104 tablet_replica.cc:330] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: stopping tablet replica
I20260806 22:42:41.444176 28104 raft_consensus.cc:2229] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:41.444211 28104 raft_consensus.cc:2258] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.444463 28104 ts_tablet_manager.cc:1905] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:41.446312 28104 ts_tablet_manager.cc:1918] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:41.446375 28104 log.cc:1196] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/f7ef169792a64555bd88a3eb4712e926
I20260806 22:42:41.446688 28104 ts_tablet_manager.cc:1939] T f7ef169792a64555bd88a3eb4712e926 P 91f23da657e8463fb5bfc2c487aa2013: Deleting consensus metadata
I20260806 22:42:41.447005 27505 catalog_manager.cc:4841] TS 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195): tablet f7ef169792a64555bd88a3eb4712e926 (table test-table [id=5fd74bbb49ed49f391b42c4f4de11553]) successfully deleted
I20260806 22:42:41.447381 28102 tablet_replica.cc:330] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: stopping tablet replica
I20260806 22:42:41.447489 28102 raft_consensus.cc:2229] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:41.447614 28102 raft_consensus.cc:2258] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.447898 28102 ts_tablet_manager.cc:1905] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:41.449856 28102 ts_tablet_manager.cc:1918] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:41.449977 28102 log.cc:1196] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/f7ef169792a64555bd88a3eb4712e926
I20260806 22:42:41.450292 28102 ts_tablet_manager.cc:1939] T f7ef169792a64555bd88a3eb4712e926 P 1970a3f92238444d9b96617fe1f695c6: Deleting consensus metadata
I20260806 22:42:41.450405 28103 tablet_replica.cc:330] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: stopping tablet replica
I20260806 22:42:41.450471 28103 raft_consensus.cc:2229] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:41.450519 28103 raft_consensus.cc:2258] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.450773 27502 catalog_manager.cc:4841] TS 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253): tablet f7ef169792a64555bd88a3eb4712e926 (table test-table [id=5fd74bbb49ed49f391b42c4f4de11553]) successfully deleted
I20260806 22:42:41.450925 28103 ts_tablet_manager.cc:1905] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:41.452725 28103 ts_tablet_manager.cc:1918] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:41.452796 28103 log.cc:1196] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/f7ef169792a64555bd88a3eb4712e926
I20260806 22:42:41.453049 28103 ts_tablet_manager.cc:1939] T f7ef169792a64555bd88a3eb4712e926 P 3ad4dc547d9741a9a590872560be60ef: Deleting consensus metadata
I20260806 22:42:41.453413 27505 catalog_manager.cc:4841] TS 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493): tablet f7ef169792a64555bd88a3eb4712e926 (table test-table [id=5fd74bbb49ed49f391b42c4f4de11553]) successfully deleted
Aug 06 22:42:41 dist-test-slave-pgkr krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:42:41 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:41 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:42:41.471796 28112 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:41.465617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.19.74.254:37569)
0806 22:42:41.465737 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.465740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.465792 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:42:41.465874 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.465874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.465884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.465941 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:41.466716 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.467226 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.468319 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.468500 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.468567 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.471737 (+ 3170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52018: BlockingRecv error: recv got EOF from 127.0.0.1:52018 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
Aug 06 22:42:41 dist-test-slave-pgkr krb5kdc[27473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM
I20260806 22:42:41.480536 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.472650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.19.74.254:37569)
0806 22:42:41.472688 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.472690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.472952 (+ 262us) server_negotiation.cc:407] Connection header received
0806 22:42:41.472981 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.472982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.472993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.473070 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:41.473506 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.474238 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.474984 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.475314 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.475435 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.477456 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:41.477457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:41.477457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:41.477465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:41.479264 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:41.479684 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:41.479686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:41.479686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:41.479757 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:41.479956 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:41.479957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:41.479958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:41.480110 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:41.480221 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.480364 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.480448 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:41.482082 27517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:41.482270 27517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:41.483785 27658 tablet_service.cc:1467] Processing CreateTablet for tablet 26b764538c6145dbbee1ed4b20abe7f8 (DEFAULT_TABLE table=test-table [id=ae9f6c346da74e4483d4fb83b2fbfd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:41.483857 27942 tablet_service.cc:1467] Processing CreateTablet for tablet 26b764538c6145dbbee1ed4b20abe7f8 (DEFAULT_TABLE table=test-table [id=ae9f6c346da74e4483d4fb83b2fbfd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:41.483855 27801 tablet_service.cc:1467] Processing CreateTablet for tablet 26b764538c6145dbbee1ed4b20abe7f8 (DEFAULT_TABLE table=test-table [id=ae9f6c346da74e4483d4fb83b2fbfd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:41.483952 27658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b764538c6145dbbee1ed4b20abe7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:41.483994 27942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b764538c6145dbbee1ed4b20abe7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:41.483994 27801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b764538c6145dbbee1ed4b20abe7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:41.485404 28119 tablet_bootstrap.cc:492] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: Bootstrap starting.
I20260806 22:42:41.485875 28119 tablet_bootstrap.cc:654] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:41.485870 28120 tablet_bootstrap.cc:492] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: Bootstrap starting.
I20260806 22:42:41.486198 28118 tablet_bootstrap.cc:492] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: Bootstrap starting.
I20260806 22:42:41.486600 28120 tablet_bootstrap.cc:654] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:41.486600 28118 tablet_bootstrap.cc:654] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:41.487128 28119 tablet_bootstrap.cc:492] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: No bootstrap required, opened a new log
I20260806 22:42:41.487200 28119 ts_tablet_manager.cc:1397] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:41.487331 28119 raft_consensus.cc:348] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.487427 28119 raft_consensus.cc:374] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:41.487455 28119 raft_consensus.cc:729] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1970a3f92238444d9b96617fe1f695c6, State: Initialized, Role: FOLLOWER
I20260806 22:42:41.487520 28120 tablet_bootstrap.cc:492] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: No bootstrap required, opened a new log
I20260806 22:42:41.487524 28119 consensus_queue.cc:260] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.487569 28120 ts_tablet_manager.cc:1397] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:42:41.487736 28120 raft_consensus.cc:348] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.487808 28120 raft_consensus.cc:374] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:41.487831 28120 raft_consensus.cc:729] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f23da657e8463fb5bfc2c487aa2013, State: Initialized, Role: FOLLOWER
I20260806 22:42:41.487891 28120 consensus_queue.cc:260] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.488117 28119 ts_tablet_manager.cc:1428] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:42:41.488165 28120 ts_tablet_manager.cc:1428] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:41.488438 28118 tablet_bootstrap.cc:492] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: No bootstrap required, opened a new log
I20260806 22:42:41.488487 28118 ts_tablet_manager.cc:1397] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260806 22:42:41.488835 28118 raft_consensus.cc:348] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.488917 28118 raft_consensus.cc:374] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:41.488945 28118 raft_consensus.cc:729] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ad4dc547d9741a9a590872560be60ef, State: Initialized, Role: FOLLOWER
I20260806 22:42:41.489015 28118 consensus_queue.cc:260] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.489177 28118 ts_tablet_manager.cc:1428] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:41.518944 28097 raft_consensus.cc:482] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:41.519033 28097 raft_consensus.cc:504] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.519284 28097 leader_election.cc:288] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:41.519481 27679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b764538c6145dbbee1ed4b20abe7f8" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f23da657e8463fb5bfc2c487aa2013" is_pre_election: true
I20260806 22:42:41.519567 27679 raft_consensus.cc:2454] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 0.
I20260806 22:42:41.519500 27961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b764538c6145dbbee1ed4b20abe7f8" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef" is_pre_election: true
I20260806 22:42:41.519608 27961 raft_consensus.cc:2454] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 0.
I20260806 22:42:41.519716 27754 leader_election.cc:302] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1970a3f92238444d9b96617fe1f695c6, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:41.519814 28097 raft_consensus.cc:2781] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:41.519883 28097 raft_consensus.cc:482] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:41.519909 28097 raft_consensus.cc:3037] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.520812 28097 raft_consensus.cc:504] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.520951 28097 leader_election.cc:288] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 election: Requested vote from peers 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:41.521113 27679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b764538c6145dbbee1ed4b20abe7f8" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f23da657e8463fb5bfc2c487aa2013"
I20260806 22:42:41.521119 27961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b764538c6145dbbee1ed4b20abe7f8" candidate_uuid: "1970a3f92238444d9b96617fe1f695c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef"
I20260806 22:42:41.521193 27961 raft_consensus.cc:3037] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.521198 27679 raft_consensus.cc:3037] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.521790 27679 raft_consensus.cc:2454] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 1.
I20260806 22:42:41.521801 27961 raft_consensus.cc:2454] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1970a3f92238444d9b96617fe1f695c6 in term 1.
I20260806 22:42:41.521956 27754 leader_election.cc:302] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1970a3f92238444d9b96617fe1f695c6, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:41.522055 28097 raft_consensus.cc:2781] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:41.522115 28097 raft_consensus.cc:686] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 1 LEADER]: Becoming Leader. State: Replica: 1970a3f92238444d9b96617fe1f695c6, State: Running, Role: LEADER
I20260806 22:42:41.522176 28097 consensus_queue.cc:237] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.522784 27517 catalog_manager.cc:5495] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194). New cstate: current_term: 1 leader_uuid: "1970a3f92238444d9b96617fe1f695c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:41.534024 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.531660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.19.74.194:34253)
0806 22:42:41.531709 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.531711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.531718 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:41.531765 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.531766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.531778 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.531819 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:41.531911 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.532579 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.533317 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.533620 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.533692 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.533807 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.533841 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.533888 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.533930 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.533952 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:41.534854 27679 raft_consensus.cc:1261] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Refusing update from remote peer 1970a3f92238444d9b96617fe1f695c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:41.534864 27961 raft_consensus.cc:1261] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Refusing update from remote peer 1970a3f92238444d9b96617fe1f695c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:41.535058 28097 consensus_queue.cc:1035] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:41.535141 28055 consensus_queue.cc:1035] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:41.537420 27518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:41.537509 27518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:41.538198 27518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: Sending DeleteTablet for 3 replicas of tablet 26b764538c6145dbbee1ed4b20abe7f8
I20260806 22:42:41.538450 27658 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b764538c6145dbbee1ed4b20abe7f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59982
I20260806 22:42:41.538455 27801 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b764538c6145dbbee1ed4b20abe7f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:53986
I20260806 22:42:41.538491 27942 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b764538c6145dbbee1ed4b20abe7f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59040
I20260806 22:42:41.538538 28104 tablet_replica.cc:330] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: stopping tablet replica
I20260806 22:42:41.538538 28102 tablet_replica.cc:330] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: stopping tablet replica
I20260806 22:42:41.538585 28102 raft_consensus.cc:2229] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:41.538586 28104 raft_consensus.cc:2229] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:41.538658 28104 raft_consensus.cc:2258] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.538661 28102 raft_consensus.cc:2258] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.538784 28103 tablet_replica.cc:330] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: stopping tablet replica
I20260806 22:42:41.538842 28103 raft_consensus.cc:2229] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:41.538890 28103 raft_consensus.cc:2258] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.539021 28104 ts_tablet_manager.cc:1905] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:41.539129 28102 ts_tablet_manager.cc:1905] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:41.540220 27518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:41.540381 27518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:41.540522 28104 ts_tablet_manager.cc:1918] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:41.540519 28102 ts_tablet_manager.cc:1918] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:41.540580 28102 log.cc:1196] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/26b764538c6145dbbee1ed4b20abe7f8
I20260806 22:42:41.540580 28104 log.cc:1196] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/26b764538c6145dbbee1ed4b20abe7f8
I20260806 22:42:41.540731 28103 ts_tablet_manager.cc:1905] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:41.540805 28104 ts_tablet_manager.cc:1939] T 26b764538c6145dbbee1ed4b20abe7f8 P 91f23da657e8463fb5bfc2c487aa2013: Deleting consensus metadata
I20260806 22:42:41.540830 28102 ts_tablet_manager.cc:1939] T 26b764538c6145dbbee1ed4b20abe7f8 P 1970a3f92238444d9b96617fe1f695c6: Deleting consensus metadata
I20260806 22:42:41.541136 27505 catalog_manager.cc:4841] TS 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195): tablet 26b764538c6145dbbee1ed4b20abe7f8 (table test-table [id=ae9f6c346da74e4483d4fb83b2fbfd4d]) successfully deleted
I20260806 22:42:41.541144 27502 catalog_manager.cc:4841] TS 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253): tablet 26b764538c6145dbbee1ed4b20abe7f8 (table test-table [id=ae9f6c346da74e4483d4fb83b2fbfd4d]) successfully deleted
I20260806 22:42:41.542043 27658 tablet_service.cc:1467] Processing CreateTablet for tablet 0028170c1d824043bb2f570fd8aa2a20 (DEFAULT_TABLE table=test-table [id=4372bb6a3ddb46f1a3b79dcecf0559af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:41.542052 27801 tablet_service.cc:1467] Processing CreateTablet for tablet 0028170c1d824043bb2f570fd8aa2a20 (DEFAULT_TABLE table=test-table [id=4372bb6a3ddb46f1a3b79dcecf0559af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:41.542063 27942 tablet_service.cc:1467] Processing CreateTablet for tablet 0028170c1d824043bb2f570fd8aa2a20 (DEFAULT_TABLE table=test-table [id=4372bb6a3ddb46f1a3b79dcecf0559af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:41.542153 27658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0028170c1d824043bb2f570fd8aa2a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:41.542160 27801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0028170c1d824043bb2f570fd8aa2a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:41.542232 27942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0028170c1d824043bb2f570fd8aa2a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:41.542237 28103 ts_tablet_manager.cc:1918] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:41.542488 28103 log.cc:1196] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/26b764538c6145dbbee1ed4b20abe7f8
I20260806 22:42:41.542726 28103 ts_tablet_manager.cc:1939] T 26b764538c6145dbbee1ed4b20abe7f8 P 3ad4dc547d9741a9a590872560be60ef: Deleting consensus metadata
I20260806 22:42:41.543123 27505 catalog_manager.cc:4841] TS 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493): tablet 26b764538c6145dbbee1ed4b20abe7f8 (table test-table [id=ae9f6c346da74e4483d4fb83b2fbfd4d]) successfully deleted
I20260806 22:42:41.543417 28118 tablet_bootstrap.cc:492] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: Bootstrap starting.
I20260806 22:42:41.543464 28119 tablet_bootstrap.cc:492] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: Bootstrap starting.
I20260806 22:42:41.543629 28120 tablet_bootstrap.cc:492] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013: Bootstrap starting.
I20260806 22:42:41.543814 28119 tablet_bootstrap.cc:654] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:41.543830 28118 tablet_bootstrap.cc:654] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:41.543959 28120 tablet_bootstrap.cc:654] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:41.544401 28119 tablet_bootstrap.cc:492] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: No bootstrap required, opened a new log
I20260806 22:42:41.544459 28119 ts_tablet_manager.cc:1397] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:41.544526 28118 tablet_bootstrap.cc:492] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: No bootstrap required, opened a new log
I20260806 22:42:41.544574 28118 ts_tablet_manager.cc:1397] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:41.544562 28119 raft_consensus.cc:348] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.544625 28119 raft_consensus.cc:374] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:41.544659 28119 raft_consensus.cc:729] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1970a3f92238444d9b96617fe1f695c6, State: Initialized, Role: FOLLOWER
I20260806 22:42:41.544706 28119 consensus_queue.cc:260] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.544734 28118 raft_consensus.cc:348] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.544793 28118 raft_consensus.cc:374] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:41.544808 28119 ts_tablet_manager.cc:1428] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:41.544823 28118 raft_consensus.cc:729] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ad4dc547d9741a9a590872560be60ef, State: Initialized, Role: FOLLOWER
I20260806 22:42:41.544866 28118 consensus_queue.cc:260] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.545094 28118 ts_tablet_manager.cc:1428] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:41.545286 28120 tablet_bootstrap.cc:492] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013: No bootstrap required, opened a new log
I20260806 22:42:41.545343 28120 ts_tablet_manager.cc:1397] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:42:41.545462 28120 raft_consensus.cc:348] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.545531 28120 raft_consensus.cc:374] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:41.545554 28120 raft_consensus.cc:729] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f23da657e8463fb5bfc2c487aa2013, State: Initialized, Role: FOLLOWER
I20260806 22:42:41.545614 28120 consensus_queue.cc:260] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.545759 28120 ts_tablet_manager.cc:1428] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:41.593060 28081 raft_consensus.cc:482] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:41.593159 28081 raft_consensus.cc:504] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.593334 28081 leader_election.cc:288] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:41.593510 27961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0028170c1d824043bb2f570fd8aa2a20" candidate_uuid: "91f23da657e8463fb5bfc2c487aa2013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef" is_pre_election: true
I20260806 22:42:41.593500 27821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0028170c1d824043bb2f570fd8aa2a20" candidate_uuid: "91f23da657e8463fb5bfc2c487aa2013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1970a3f92238444d9b96617fe1f695c6" is_pre_election: true
I20260806 22:42:41.593627 27821 raft_consensus.cc:2454] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f23da657e8463fb5bfc2c487aa2013 in term 0.
I20260806 22:42:41.593633 27961 raft_consensus.cc:2454] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f23da657e8463fb5bfc2c487aa2013 in term 0.
I20260806 22:42:41.593802 27612 leader_election.cc:302] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ad4dc547d9741a9a590872560be60ef, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:41.593932 28081 raft_consensus.cc:2781] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:41.593988 28081 raft_consensus.cc:482] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:41.594007 28081 raft_consensus.cc:3037] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.594753 28081 raft_consensus.cc:504] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.594899 28081 leader_election.cc:288] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [CANDIDATE]: Term 1 election: Requested vote from peers 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253), 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493)
I20260806 22:42:41.595036 27821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0028170c1d824043bb2f570fd8aa2a20" candidate_uuid: "91f23da657e8463fb5bfc2c487aa2013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1970a3f92238444d9b96617fe1f695c6"
I20260806 22:42:41.595105 27961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0028170c1d824043bb2f570fd8aa2a20" candidate_uuid: "91f23da657e8463fb5bfc2c487aa2013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad4dc547d9741a9a590872560be60ef"
I20260806 22:42:41.595142 27821 raft_consensus.cc:3037] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.595178 27961 raft_consensus.cc:3037] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:41.595930 27961 raft_consensus.cc:2454] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f23da657e8463fb5bfc2c487aa2013 in term 1.
I20260806 22:42:41.595929 27821 raft_consensus.cc:2454] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f23da657e8463fb5bfc2c487aa2013 in term 1.
I20260806 22:42:41.596112 27612 leader_election.cc:302] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ad4dc547d9741a9a590872560be60ef, 91f23da657e8463fb5bfc2c487aa2013; no voters:
I20260806 22:42:41.596230 28081 raft_consensus.cc:2781] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:41.596280 28081 raft_consensus.cc:686] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 LEADER]: Becoming Leader. State: Replica: 91f23da657e8463fb5bfc2c487aa2013, State: Running, Role: LEADER
I20260806 22:42:41.596333 28081 consensus_queue.cc:237] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } }
I20260806 22:42:41.596959 27518 catalog_manager.cc:5495] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "91f23da657e8463fb5bfc2c487aa2013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f23da657e8463fb5bfc2c487aa2013" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:41.608866 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.606065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.19.74.193:36195)
0806 22:42:41.606108 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.606110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.606119 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:41.606196 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.606197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.606210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.606281 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:41.606433 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.607333 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.608054 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.608382 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.608448 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.608592 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.608639 (+ 47us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.608694 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.608739 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.608766 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:41.613749 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.611122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:33989 (local address 127.19.74.193:36195)
0806 22:42:41.611160 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:41.611161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:41.611193 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:42:41.611216 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:41.611216 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:41.611227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:41.611288 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:41.611359 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.612128 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.612998 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.613238 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.613300 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.613480 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.613538 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.613618 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:41.613686 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:42:41.613705 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":18}
I20260806 22:42:41.613879 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.611051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36195 (local address 127.19.74.193:33989)
0806 22:42:41.611103 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.611114 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:41.611153 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:41.611288 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:41.611289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:41.611290 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:42:41.611319 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.611320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.612213 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.612213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.612915 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:41.612919 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:41.613325 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:41.613326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:41.613394 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:41.613401 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:42:41.613627 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:42:41.613629 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:42:41.613684 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:42:41.613745 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:42:41.615773 27821 raft_consensus.cc:1261] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Refusing update from remote peer 91f23da657e8463fb5bfc2c487aa2013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:41.615922 27961 raft_consensus.cc:1261] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Refusing update from remote peer 91f23da657e8463fb5bfc2c487aa2013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:41.615965 28092 consensus_queue.cc:1035] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1970a3f92238444d9b96617fe1f695c6" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 34253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:41.616106 28092 consensus_queue.cc:1035] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ad4dc547d9741a9a590872560be60ef" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:41.628046 27517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:41.628144 27517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:41.628798 27517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22e4d688d24142b88a2d7b4f717b92c4: Sending DeleteTablet for 3 replicas of tablet 0028170c1d824043bb2f570fd8aa2a20
I20260806 22:42:41.629034 27658 tablet_service.cc:1514] Processing DeleteTablet for tablet 0028170c1d824043bb2f570fd8aa2a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59982
I20260806 22:42:41.629118 28104 tablet_replica.cc:330] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013: stopping tablet replica
I20260806 22:42:41.629094 27942 tablet_service.cc:1514] Processing DeleteTablet for tablet 0028170c1d824043bb2f570fd8aa2a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:59040
I20260806 22:42:41.629161 28104 raft_consensus.cc:2229] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:41.629182 28103 tablet_replica.cc:330] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: stopping tablet replica
I20260806 22:42:41.629242 28104 raft_consensus.cc:2258] T 0028170c1d824043bb2f570fd8aa2a20 P 91f23da657e8463fb5bfc2c487aa2013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.629238 28103 raft_consensus.cc:2229] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:41.629276 28103 raft_consensus.cc:2258] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.629484 27801 tablet_service.cc:1514] Processing DeleteTablet for tablet 0028170c1d824043bb2f570fd8aa2a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:41 UTC) from {username='oryx', principal='oryx/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:53986
I20260806 22:42:41.629578 28102 tablet_replica.cc:330] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: stopping tablet replica
I20260806 22:42:41.629648 28102 raft_consensus.cc:2229] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:41.629690 28102 raft_consensus.cc:2258] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:41.629971 28102 ts_tablet_manager.cc:1905] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:41.630156 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 27591
I20260806 22:42:41.631635 28102 ts_tablet_manager.cc:1918] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:41.631690 28102 log.cc:1196] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/0028170c1d824043bb2f570fd8aa2a20
I20260806 22:42:41.631927 28102 ts_tablet_manager.cc:1939] T 0028170c1d824043bb2f570fd8aa2a20 P 1970a3f92238444d9b96617fe1f695c6: Deleting consensus metadata
I20260806 22:42:41.632395 27502 catalog_manager.cc:4841] TS 1970a3f92238444d9b96617fe1f695c6 (127.19.74.194:34253): tablet 0028170c1d824043bb2f570fd8aa2a20 (table test-table [id=4372bb6a3ddb46f1a3b79dcecf0559af]) successfully deleted
I20260806 22:42:41.634861 28103 ts_tablet_manager.cc:1905] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:41.636155 28103 ts_tablet_manager.cc:1918] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:42:41.636215 28103 log.cc:1196] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/0028170c1d824043bb2f570fd8aa2a20
I20260806 22:42:41.636435 28103 ts_tablet_manager.cc:1939] T 0028170c1d824043bb2f570fd8aa2a20 P 3ad4dc547d9741a9a590872560be60ef: Deleting consensus metadata
I20260806 22:42:41.636770 27505 catalog_manager.cc:4841] TS 3ad4dc547d9741a9a590872560be60ef (127.19.74.195:37493): tablet 0028170c1d824043bb2f570fd8aa2a20 (table test-table [id=4372bb6a3ddb46f1a3b79dcecf0559af]) successfully deleted
W20260806 22:42:41.640545 27505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:36195 (error 108)
I20260806 22:42:41.640827 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 27735
I20260806 22:42:41.641784 28137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.640944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36195 (local address 127.0.0.1:60014)
0806 22:42:41.641719 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.641754 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:36195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":39,"threads_started":1}
W20260806 22:42:41.641891 27505 catalog_manager.cc:4584] TS 91f23da657e8463fb5bfc2c487aa2013 (127.19.74.193:36195): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0028170c1d824043bb2f570fd8aa2a20: Network error: Client connection negotiation failed: client connection to 127.19.74.193:36195: connect: Connection refused (error 111)
I20260806 22:42:41.648197 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 27876
I20260806 22:42:41.654392 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 27486
2026-08-06T22:42:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2746 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:41 dist-test-slave-pgkr krb5kdc[28140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:41 dist-test-slave-pgkr krb5kdc[28140](info): set up 2 sockets
Aug 06 22:42:41 dist-test-slave-pgkr krb5kdc[28140](info): commencing operation
krb5kdc: starting...
W20260806 22:42:42.705317 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:42 dist-test-slave-pgkr krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:42.824740 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:41321
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:41321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:42.905462 28154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:42.905669 28154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:42.905691 28154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:42.906729 28154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:42.906760 28154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:42.906773 28154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:42.906785 28154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:42.906796 28154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:42.908180 28154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:41321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:41321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:42.908390 28154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:42.908556 28154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:42.913527 28160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:42.913539 28159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:42.913558 28162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:42.913959 28154 server_base.cc:1034] running on GCE node
I20260806 22:42:42.914222 28154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:42.914458 28154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:42.915594 28154 hybrid_clock.cc:648] HybridClock initialized: now 1786056162915581 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T22:42:42Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1265 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:42 dist-test-slave-pgkr krb5kdc[28167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:42 dist-test-slave-pgkr krb5kdc[28167](info): set up 2 sockets
Aug 06 22:42:42 dist-test-slave-pgkr krb5kdc[28167](info): commencing operation
krb5kdc: starting...
W20260806 22:42:43.972919 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:42:44.089355 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:32953
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:32953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:44.170792 28180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:44.171010 28180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:44.171043 28180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:44.172125 28180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:44.172195 28180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:44.172219 28180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:44.172266 28180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:44.172318 28180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:44.173763 28180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:32953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:32953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:44.174029 28180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:44.174255 28180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:44.178978 28185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:44.178989 28186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:44.179013 28188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:44.179248 28180 server_base.cc:1034] running on GCE node
I20260806 22:42:44.179553 28180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:44.179840 28180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:44.181001 28180 hybrid_clock.cc:648] HybridClock initialized: now 1786056164180987 us; error 25 us; skew 500 ppm
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:44.183349 28180 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:44.183758 28180 webserver.cc:460] Webserver started at http://127.19.74.254:45253/ using document root <none> and password file <none>
I20260806 22:42:44.183970 28180 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:44.184010 28180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:44.184095 28180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:44.185038 28180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "e82526ebb06e4fdea5fa426512913f3e"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "368fed6d9213b20756740de733b2b600"
server_key_iv: "99752047763b32036ddfc992ff85ae96"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.185338 28180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "e82526ebb06e4fdea5fa426512913f3e"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "368fed6d9213b20756740de733b2b600"
server_key_iv: "99752047763b32036ddfc992ff85ae96"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.186720 28180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:42:44.187422 28195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.187615 28180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:44.187688 28180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "e82526ebb06e4fdea5fa426512913f3e"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "368fed6d9213b20756740de733b2b600"
server_key_iv: "99752047763b32036ddfc992ff85ae96"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.187785 28180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:44.199394 28180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:44.201360 28180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:44.201526 28180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:44.206007 28180 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:32953
I20260806 22:42:44.206030 28248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:32953 every 8 connection(s)
I20260806 22:42:44.206418 28180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:44.207373 28249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:44.209729 28249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Bootstrap starting.
I20260806 22:42:44.210343 28249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:44.210701 28249 log.cc:824] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:44.211460 28249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: No bootstrap required, opened a new log
I20260806 22:42:44.213065 28249 raft_consensus.cc:348] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } }
I20260806 22:42:44.213217 28249 raft_consensus.cc:374] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:44.213241 28249 raft_consensus.cc:729] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82526ebb06e4fdea5fa426512913f3e, State: Initialized, Role: FOLLOWER
I20260806 22:42:44.213409 28249 consensus_queue.cc:260] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } }
I20260806 22:42:44.213472 28249 raft_consensus.cc:388] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:44.213529 28249 raft_consensus.cc:482] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:44.213577 28249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:44.213830 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 28180
I20260806 22:42:44.213956 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:42:44.214159 19755 external_mini_cluster.cc:1428] Setting key 1ca5c747b839982d7c5e27cd19989c2a
I20260806 22:42:44.214282 28249 raft_consensus.cc:504] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } }
I20260806 22:42:44.214402 28249 leader_election.cc:302] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e82526ebb06e4fdea5fa426512913f3e; no voters:
I20260806 22:42:44.214564 28249 leader_election.cc:288] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:44.214836 28254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:44.214946 28249 sys_catalog.cc:564] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:44.215062 28254 raft_consensus.cc:686] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 1 LEADER]: Becoming Leader. State: Replica: e82526ebb06e4fdea5fa426512913f3e, State: Running, Role: LEADER
I20260806 22:42:44.215221 28254 consensus_queue.cc:237] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } }
I20260806 22:42:44.215637 28256 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e82526ebb06e4fdea5fa426512913f3e. Latest consensus state: current_term: 1 leader_uuid: "e82526ebb06e4fdea5fa426512913f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } } }
I20260806 22:42:44.215627 28252 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e82526ebb06e4fdea5fa426512913f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } } }
I20260806 22:42:44.215785 28252 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:44.215741 28256 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:44.216571 28262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:44.217202 28262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:44.218708 28262 catalog_manager.cc:1347] Generated new cluster ID: d8d9d3071575471ebb911191d5e22cc6
I20260806 22:42:44.218770 28262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:44.227751 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.214793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.19.74.254:32953)
0806 22:42:44.215610 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:44.215620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:44.215657 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:42:44.215779 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:44.215787 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:44.215846 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:44.215991 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:44.217073 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.218510 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.219640 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.220247 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.220349 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:44.223221 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:44.223225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:44.223226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:44.223238 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:44.225832 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:44.226247 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:44.226250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:44.226250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:44.226362 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:44.226598 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:44.226598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:44.226599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:44.227022 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:44.227182 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:44.227294 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:42:44.227523 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":87,"threads_started":1}
I20260806 22:42:44.247007 28262 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:44.247561 28262 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:44.251982 28262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Generated new TSK 0
I20260806 22:42:44.252161 28262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:42:44.282197 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:32953
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:44.364037 28275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:44.364228 28275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:44.364250 28275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:44.365360 28275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:44.365408 28275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:44.365460 28275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:44.367010 28275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:32953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:44.367275 28275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:44.367496 28275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:44.372393 28280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:44.372406 28283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:44.372463 28275 server_base.cc:1034] running on GCE node
W20260806 22:42:44.372402 28281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:44.372821 28275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:44.373055 28275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:44.374218 28275 hybrid_clock.cc:648] HybridClock initialized: now 1786056164374197 us; error 32 us; skew 500 ppm
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:44.376825 28275 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:42:44.377257 28275 webserver.cc:460] Webserver started at http://127.19.74.193:43341/ using document root <none> and password file <none>
I20260806 22:42:44.377485 28275 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:44.377552 28275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:44.377707 28275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:44.378760 28275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "786b915e7a3745f99471b0108daf7fe4"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "178ba6212879726f93e7d78129823260"
server_key_iv: "22dddd1176703a6e45be13c8d8c048ac"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.379092 28275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "786b915e7a3745f99471b0108daf7fe4"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "178ba6212879726f93e7d78129823260"
server_key_iv: "22dddd1176703a6e45be13c8d8c048ac"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.380431 28275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:44.381155 28291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.381327 28275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:44.381433 28275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "786b915e7a3745f99471b0108daf7fe4"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "178ba6212879726f93e7d78129823260"
server_key_iv: "22dddd1176703a6e45be13c8d8c048ac"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.381505 28275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:44.407850 28275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:44.410079 28275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:44.410233 28275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:44.410446 28275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:44.410799 28275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:44.410849 28275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.410882 28275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:44.410908 28275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.416721 28275 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:36581
I20260806 22:42:44.416792 28406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:36581 every 8 connection(s)
I20260806 22:42:44.417136 28275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:44.425767 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.417881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:41247 (local address 127.19.74.254:32953)
0806 22:42:44.417949 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:44.417951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:44.418164 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:42:44.418650 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:44.418651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:44.418663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:44.418747 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:44.419602 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.420112 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.421047 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.421245 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.421301 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:44.423406 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:44.423408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:44.423408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:44.423415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:44.424653 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:44.425000 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:44.425001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:44.425001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:44.425056 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:44.425270 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:44.425270 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:44.425270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:44.425366 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:44.425416 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:44.425636 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 22:42:44.425704 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:42:44.425942 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.417870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:41247)
0806 22:42:44.418032 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:44.418073 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:44.418527 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:44.418797 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:44.418800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:44.419335 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:44.419515 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:44.419519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.420192 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.420192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:44.420958 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:44.420960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.421321 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.421322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:44.421421 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:44.421920 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:44.421929 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:44.423317 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:44.424723 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:44.424729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:44.424731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:44.424932 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:44.425117 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:44.425118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:44.425119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:44.425214 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:44.425420 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:44.425424 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:44.425529 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 22:42:44.425613 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:42:44.425724 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260806 22:42:44.426501 28407 heartbeater.cc:347] Connected to a master server at 127.19.74.254:32953
I20260806 22:42:44.426594 28407 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:44.426787 28407 heartbeater.cc:511] Master 127.19.74.254:32953 requested a full tablet report, sending...
I20260806 22:42:44.427332 28212 ts_manager.cc:194] Registered new tserver with Master: 786b915e7a3745f99471b0108daf7fe4 (127.19.74.193:36581)
I20260806 22:42:44.427376 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 28275
I20260806 22:42:44.427481 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:44.427700 19755 external_mini_cluster.cc:1428] Setting key 3da18c0b02535845b9cdfdab03a8184a
I20260806 22:42:44.428215 28212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:41247
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:42:44.451125 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:32953
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:44.530594 28412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:44.530833 28412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:44.530869 28412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:44.531976 28412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:44.532045 28412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:44.532090 28412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:44.533495 28412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:32953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:44.533774 28412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:44.534015 28412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:44.538882 28418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:44.538897 28419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:44.538897 28421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:44.539264 28412 server_base.cc:1034] running on GCE node
I20260806 22:42:44.539494 28412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:44.539731 28412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:44.540894 28412 hybrid_clock.cc:648] HybridClock initialized: now 1786056164540879 us; error 48 us; skew 500 ppm
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:44.543418 28412 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:42:44.543850 28412 webserver.cc:460] Webserver started at http://127.19.74.194:43641/ using document root <none> and password file <none>
I20260806 22:42:44.544102 28412 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:44.544173 28412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:44.544322 28412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:44.545478 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "4d39648493034f71b50274db60d6cda9"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "a4062c73de6d15126c90ce2b52cd88d9"
server_key_iv: "23350fd317ae902181767ffd26b25a90"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.545835 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "4d39648493034f71b50274db60d6cda9"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "a4062c73de6d15126c90ce2b52cd88d9"
server_key_iv: "23350fd317ae902181767ffd26b25a90"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.547217 28412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:42:44.547977 28428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.548187 28412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:44.548295 28412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "4d39648493034f71b50274db60d6cda9"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "a4062c73de6d15126c90ce2b52cd88d9"
server_key_iv: "23350fd317ae902181767ffd26b25a90"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.548368 28412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:44.571233 28412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:44.573055 28412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:44.573215 28412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:44.573453 28412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:44.573772 28412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:44.573827 28412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.573868 28412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:44.573894 28412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.579939 28412 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:37759
I20260806 22:42:44.579985 28543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:37759 every 8 connection(s)
I20260806 22:42:44.580281 28412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:44.585793 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 28412
I20260806 22:42:44.585916 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:44.586138 19755 external_mini_cluster.cc:1428] Setting key 8e2c0659f4473f3846bae40178e7a2f3
I20260806 22:42:44.589442 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.580985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:40437 (local address 127.19.74.254:32953)
0806 22:42:44.581048 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:44.581050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:44.581242 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:42:44.581701 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:44.581702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:44.581711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:44.581771 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:44.582716 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.583183 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.584154 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.584411 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.584465 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:44.586567 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:44.586569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:44.586569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:44.586577 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:44.588029 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:44.588463 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:44.588464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:44.588465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:44.588530 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:44.588805 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:44.588806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:44.588806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:44.588922 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:44.588997 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:44.589270 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 22:42:44.589351 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:42:44.589699 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.580962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:40437)
0806 22:42:44.581115 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:44.581156 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:44.581621 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:44.581823 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:44.581826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:44.582402 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:44.582599 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:44.582603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.583256 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.583257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:44.584078 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:44.584087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.584484 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.584485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:44.584580 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:44.584992 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:44.585000 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:44.586465 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:44.588142 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:44.588149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:44.588151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:44.588379 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:44.588605 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:44.588606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:44.588613 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:44.588721 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:44.588988 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:44.588995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:44.589135 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 22:42:44.589247 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:42:44.589404 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260806 22:42:44.590157 28544 heartbeater.cc:347] Connected to a master server at 127.19.74.254:32953
I20260806 22:42:44.590265 28544 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:44.590437 28544 heartbeater.cc:511] Master 127.19.74.254:32953 requested a full tablet report, sending...
I20260806 22:42:44.590862 28212 ts_manager.cc:194] Registered new tserver with Master: 4d39648493034f71b50274db60d6cda9 (127.19.74.194:37759)
I20260806 22:42:44.591419 28212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:40437
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:42:44.612715 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:32953
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:44.693639 28549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:44.693822 28549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:44.693845 28549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:44.694933 28549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:44.694972 28549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:44.694998 28549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:44.696316 28549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:32953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:44.696506 28549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:44.696669 28549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:44.701586 28557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:44.701575 28559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:44.701627 28549 server_base.cc:1034] running on GCE node
W20260806 22:42:44.701576 28555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:44.702031 28549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:44.702209 28549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:44.703338 28549 hybrid_clock.cc:648] HybridClock initialized: now 1786056164703326 us; error 29 us; skew 500 ppm
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:44.705621 28549 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:42:44.706074 28549 webserver.cc:460] Webserver started at http://127.19.74.195:40495/ using document root <none> and password file <none>
I20260806 22:42:44.706301 28549 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:44.706365 28549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:44.706507 28549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:44.707542 28549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "6aa63c489787498993abd12387ffb667"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "250e14c3a6c1691de7fe3042c5c81412"
server_key_iv: "9bb3073e5da1d81a04af8cff0307d560"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.707901 28549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "6aa63c489787498993abd12387ffb667"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "250e14c3a6c1691de7fe3042c5c81412"
server_key_iv: "9bb3073e5da1d81a04af8cff0307d560"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.709203 28549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:42:44.709887 28566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.710048 28549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:44.710137 28549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "6aa63c489787498993abd12387ffb667"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "250e14c3a6c1691de7fe3042c5c81412"
server_key_iv: "9bb3073e5da1d81a04af8cff0307d560"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.710212 28549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:44.750794 28549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:44.753063 28549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:44.753226 28549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:44.753481 28549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:44.753820 28549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:44.753876 28549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.753918 28549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:44.753943 28549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.759974 28549 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:45443
I20260806 22:42:44.760020 28681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:45443 every 8 connection(s)
I20260806 22:42:44.760344 28549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:42:44.767711 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 28549
I20260806 22:42:44.767802 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:44.768020 19755 external_mini_cluster.cc:1428] Setting key 0f243ee98ceb4337cdd41a68efe23e38
I20260806 22:42:44.769306 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.761044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:37359 (local address 127.19.74.254:32953)
0806 22:42:44.761087 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:44.761089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:44.761279 (+ 190us) server_negotiation.cc:407] Connection header received
0806 22:42:44.761804 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:44.761806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:44.761819 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:44.761864 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:44.762913 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.763393 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.764335 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.764526 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.764633 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:44.766661 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:44.766663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:44.766663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:44.766669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:44.767864 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:44.768328 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:44.768330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:44.768331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:44.768407 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:44.768677 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:44.768678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:44.768678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:44.768802 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:44.768875 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:44.769133 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 22:42:44.769209 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:44.769479 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.761011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:37359)
0806 22:42:44.761165 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:44.761200 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:44.761694 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:44.761943 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:44.761947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:44.762536 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:44.762805 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:44.762810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.763485 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.763486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:44.764238 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:44.764240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:44.764642 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:44.764644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:44.764777 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:44.765249 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:44.765258 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:44.766535 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:44.767955 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:44.767961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:44.767963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:44.768265 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:44.768483 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:44.768484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:44.768484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:44.768604 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:44.768866 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:44.768873 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:44.769003 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 22:42:44.769100 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:42:44.769242 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260806 22:42:44.769963 28682 heartbeater.cc:347] Connected to a master server at 127.19.74.254:32953
I20260806 22:42:44.770066 28682 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:44.770239 28682 heartbeater.cc:511] Master 127.19.74.254:32953 requested a full tablet report, sending...
I20260806 22:42:44.770598 28212 ts_manager.cc:194] Registered new tserver with Master: 6aa63c489787498993abd12387ffb667 (127.19.74.195:45443)
I20260806 22:42:44.770995 28212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:37359
I20260806 22:42:44.779731 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab.
Entry for principal oryx/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab.
WARNING: no policy specified for oryx/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab.
Entry for principal oryx/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab.
WARNING: no policy specified for oryx/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab.
Entry for principal oryx/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab.
I20260806 22:42:44.846197 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 28275
I20260806 22:42:44.852607 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 28412
I20260806 22:42:44.857915 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 28549
I20260806 22:42:44.862753 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 28180
I20260806 22:42:44.868614 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:32953
--webserver_interface=127.19.74.254
--webserver_port=45253
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:32953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:44.948760 28691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:44.948940 28691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:44.948961 28691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:44.949906 28691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:44.949939 28691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:44.949951 28691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:44.949962 28691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:44.949973 28691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:44.951292 28691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:32953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:32953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=45253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:44.951463 28691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:44.951611 28691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:44.956219 28697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:44.956236 28699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:44.956233 28696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:44.956254 28691 server_base.cc:1034] running on GCE node
I20260806 22:42:44.956563 28691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:44.956833 28691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:44.957974 28691 hybrid_clock.cc:648] HybridClock initialized: now 1786056164957966 us; error 31 us; skew 500 ppm
Aug 06 22:42:44 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056164, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:44.960462 28691 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:42:44.960961 28691 webserver.cc:460] Webserver started at http://127.19.74.254:45253/ using document root <none> and password file <none>
I20260806 22:42:44.961203 28691 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:44.961292 28691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:44.962744 28691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:44.963438 28706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:44.963642 28691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:42:44.963752 28691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "e82526ebb06e4fdea5fa426512913f3e"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "368fed6d9213b20756740de733b2b600"
server_key_iv: "99752047763b32036ddfc992ff85ae96"
server_key_version: "encryptionkey@0"
I20260806 22:42:44.964087 28691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:44.987452 28691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:44.989284 28691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:44.989439 28691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:44.993345 28691 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:32953
I20260806 22:42:44.993378 28762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:32953 every 8 connection(s)
I20260806 22:42:44.993765 28691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:44.994024 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 28691
I20260806 22:42:44.994392 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:36581
--local_ip_for_outbound_sockets=127.19.74.193
--tserver_master_addrs=127.19.74.254:32953
--webserver_port=43341
--webserver_interface=127.19.74.193
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.74.193
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:44.995402 28763 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:42:44.996110 28763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Bootstrap starting.
I20260806 22:42:45.006228 28763 log.cc:824] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:45.008759 28763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:42:45.009105 28763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Bootstrap complete.
I20260806 22:42:45.012346 28763 raft_consensus.cc:348] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } }
I20260806 22:42:45.012650 28763 raft_consensus.cc:729] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82526ebb06e4fdea5fa426512913f3e, State: Initialized, Role: FOLLOWER
I20260806 22:42:45.012816 28763 consensus_queue.cc:260] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } }
I20260806 22:42:45.012878 28763 raft_consensus.cc:388] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:45.012914 28763 raft_consensus.cc:482] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:45.012959 28763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:42:45.014212 28763 raft_consensus.cc:504] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } }
I20260806 22:42:45.014323 28763 leader_election.cc:302] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e82526ebb06e4fdea5fa426512913f3e; no voters:
I20260806 22:42:45.014475 28763 leader_election.cc:288] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:42:45.014542 28767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:42:45.014725 28763 sys_catalog.cc:564] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:45.014780 28767 raft_consensus.cc:686] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [term 2 LEADER]: Becoming Leader. State: Replica: e82526ebb06e4fdea5fa426512913f3e, State: Running, Role: LEADER
I20260806 22:42:45.014918 28767 consensus_queue.cc:237] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } }
I20260806 22:42:45.015262 28768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e82526ebb06e4fdea5fa426512913f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } } }
I20260806 22:42:45.015344 28768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:45.015584 28769 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e82526ebb06e4fdea5fa426512913f3e. Latest consensus state: current_term: 2 leader_uuid: "e82526ebb06e4fdea5fa426512913f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82526ebb06e4fdea5fa426512913f3e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 32953 } } }
I20260806 22:42:45.015667 28769 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:45.015704 28776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:45.016240 28776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:45.016368 28776 catalog_manager.cc:1259] Loaded cluster ID: d8d9d3071575471ebb911191d5e22cc6
I20260806 22:42:45.016405 28776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:45.017560 28776 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:45.017733 28776 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e82526ebb06e4fdea5fa426512913f3e: Loaded TSK: 0
I20260806 22:42:45.017946 28776 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:42:45.083913 28765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:45.084125 28765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:45.084160 28765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:45.085319 28765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:45.085384 28765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:45.085430 28765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:42:45.086910 28765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.193.keytab
--principal=oryx/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:36581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=43341
--tserver_master_addrs=127.19.74.254:32953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:45.087199 28765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:45.087422 28765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:45.092489 28790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:45.092533 28789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:45.092794 28792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:45.092693 28765 server_base.cc:1034] running on GCE node
I20260806 22:42:45.093003 28765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:45.093284 28765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:45.094475 28765 hybrid_clock.cc:648] HybridClock initialized: now 1786056165094446 us; error 40 us; skew 500 ppm
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056165, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:45.097471 28765 init.cc:377] Logged in from keytab as oryx/127.19.74.193@KRBTEST.COM (short username oryx)
I20260806 22:42:45.097980 28765 webserver.cc:460] Webserver started at http://127.19.74.193:43341/ using document root <none> and password file <none>
I20260806 22:42:45.098209 28765 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:45.098296 28765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:45.099764 28765 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:45.100448 28799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.100629 28765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:42:45.100736 28765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "786b915e7a3745f99471b0108daf7fe4"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "178ba6212879726f93e7d78129823260"
server_key_iv: "22dddd1176703a6e45be13c8d8c048ac"
server_key_version: "encryptionkey@0"
I20260806 22:42:45.101071 28765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:45.128105 28765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:45.130474 28765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:45.130707 28765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:45.130970 28765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:45.131341 28765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:45.131394 28765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.131438 28765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:45.131464 28765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.138171 28765 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:36581
I20260806 22:42:45.138217 28914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:36581 every 8 connection(s)
I20260806 22:42:45.138571 28765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:45.142809 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 28765
I20260806 22:42:45.143138 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:37759
--local_ip_for_outbound_sockets=127.19.74.194
--tserver_master_addrs=127.19.74.254:32953
--webserver_port=43641
--webserver_interface=127.19.74.194
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.74.194
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.148000 28917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.139275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:38945)
0806 22:42:45.139500 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.139567 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.140218 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.140776 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.140780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.141590 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.141898 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.141905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.143335 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.143336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.144107 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.144109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.144702 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.144703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.144837 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.145396 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.145404 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.147794 (+ 2390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
W20260806 22:42:45.148052 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.139409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:38945 (local address 127.19.74.254:32953)
0806 22:42:45.139737 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.139745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.139783 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:42:45.140451 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.140481 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.140539 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.140703 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.142026 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.143233 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.144210 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.144608 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.144729 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.147908 (+ 3179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:38945: BlockingRecv error: recv got EOF from 127.19.74.193:38945 (error 108)
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":101,"threads_started":1}
W20260806 22:42:45.148582 28915 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:32953 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.74.254:32953: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.153452 28917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.148850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:37629)
0806 22:42:45.148898 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.148908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.148945 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.149076 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.149077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.149231 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.149289 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.149291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.149850 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.149851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.150791 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.150795 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.151148 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.151149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.151225 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.151599 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.151605 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.153411 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:42:45.153672 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.148895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:37629 (local address 127.19.74.254:32953)
0806 22:42:45.148950 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.148952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.148985 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:42:45.149002 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.149003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.149012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.149049 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.149365 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.149780 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.150895 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.151079 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.151126 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.153586 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:37629: BlockingRecv error: recv got EOF from 127.19.74.193:37629 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.157804 28917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.153726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:50083)
0806 22:42:45.153763 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.153772 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.153804 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.153943 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.153943 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.154046 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.154069 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.154070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.154913 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.154913 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.155322 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.155324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.155768 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.155769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.155802 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.156172 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.156176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.157740 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 22:42:45.157891 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.153773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:50083 (local address 127.19.74.254:32953)
0806 22:42:45.153823 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.153825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.153836 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:45.153858 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.153859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.153868 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.153935 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.154140 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.154842 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.155392 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.155698 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.155764 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.157814 (+ 2050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:50083: BlockingRecv error: recv got EOF from 127.19.74.193:50083 (error 108)
Metrics: {"server-negotiator.queue_time_us":34,"thread_start_us":82,"threads_started":1}
W20260806 22:42:45.157933 28915 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:42:45.227288 28919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:45.227483 28919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:45.227505 28919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:45.228586 28919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:45.228619 28919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:45.228643 28919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:42:45.229955 28919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.194.keytab
--principal=oryx/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:37759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=43641
--tserver_master_addrs=127.19.74.254:32953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:45.230162 28919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:45.230341 28919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:45.235337 28928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:45.235323 28926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:45.235327 28925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:45.235399 28919 server_base.cc:1034] running on GCE node
I20260806 22:42:45.235694 28919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:45.235929 28919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:45.237087 28919 hybrid_clock.cc:648] HybridClock initialized: now 1786056165237077 us; error 40 us; skew 500 ppm
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056165, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:45.244895 28919 init.cc:377] Logged in from keytab as oryx/127.19.74.194@KRBTEST.COM (short username oryx)
I20260806 22:42:45.245436 28919 webserver.cc:460] Webserver started at http://127.19.74.194:43641/ using document root <none> and password file <none>
I20260806 22:42:45.245747 28919 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:45.245826 28919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:45.247778 28919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:45.248521 28935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.248792 28919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:45.248936 28919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "4d39648493034f71b50274db60d6cda9"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "a4062c73de6d15126c90ce2b52cd88d9"
server_key_iv: "23350fd317ae902181767ffd26b25a90"
server_key_version: "encryptionkey@0"
I20260806 22:42:45.249399 28919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:45.273097 28919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:45.275403 28919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:45.275593 28919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:45.275851 28919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:45.276188 28919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:45.276226 28919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.276283 28919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:45.276319 28919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.282737 28919 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:37759
I20260806 22:42:45.282791 29050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:37759 every 8 connection(s)
I20260806 22:42:45.283142 28919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.289323 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 28919
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.289635 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:45443
--local_ip_for_outbound_sockets=127.19.74.195
--tserver_master_addrs=127.19.74.254:32953
--webserver_port=40495
--webserver_interface=127.19.74.195
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.74.195
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:45.290515 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.283898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:38301 (local address 127.19.74.254:32953)
0806 22:42:45.283969 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.283972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.284256 (+ 284us) server_negotiation.cc:407] Connection header received
0806 22:42:45.284724 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.284726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.284741 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.284803 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.285687 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.286205 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.287199 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.287395 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.287465 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.290459 (+ 2994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:38301: BlockingRecv error: recv got EOF from 127.19.74.194:38301 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:42:45.290576 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.283899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:38301)
0806 22:42:45.284095 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.284152 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.284640 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.284875 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.284879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.285420 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.285602 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.285606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.286314 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.286314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.287107 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.287109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.287478 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.287479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.287577 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.288043 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.288051 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.290379 (+ 2328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
W20260806 22:42:45.290828 29051 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:32953 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.74.254:32953: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.295408 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.291076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:39413)
0806 22:42:45.291124 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.291133 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.291173 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.291308 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.291309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.291442 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.291481 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.291482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.292117 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.292118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.292640 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.292641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.292986 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.292987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.293049 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.293501 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.293505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.295327 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:42:45.295459 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.291126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:39413 (local address 127.19.74.254:32953)
0806 22:42:45.291184 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.291186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.291195 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:45.291221 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.291222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.291231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.291306 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.291561 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.292026 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.292722 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.292914 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.292988 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.295396 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:39413: BlockingRecv error: recv got EOF from 127.19.74.194:39413 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.299908 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.295651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:55227)
0806 22:42:45.295773 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.295782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.295831 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.295980 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.295981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.296130 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.296165 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.296166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.296770 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.296771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.297240 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.297242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.297592 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.297594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.297650 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.298074 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.298078 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.299831 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
W20260806 22:42:45.299955 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.295703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:55227 (local address 127.19.74.254:32953)
0806 22:42:45.295739 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.295741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.295840 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:42:45.295884 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.295886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.295895 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.295978 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.296243 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.296689 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.297323 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.297508 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.297597 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.299894 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:55227: BlockingRecv error: recv got EOF from 127.19.74.194:55227 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:42:45.300024 29051 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:42:45.372121 29054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:45.372362 29054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:45.372401 29054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:45.373553 29054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:45.373626 29054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:45.373672 29054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:42:45.375197 29054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/krb5kdc/oryx_127.19.74.195.keytab
--principal=oryx/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:45443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=40495
--tserver_master_addrs=127.19.74.254:32953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:45.375454 29054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:45.375690 29054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:45.380646 29063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:45.380621 29061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:45.380816 29054 server_base.cc:1034] running on GCE node
W20260806 22:42:45.380702 29060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:45.381085 29054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:45.381390 29054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:45.382551 29054 hybrid_clock.cc:648] HybridClock initialized: now 1786056165382530 us; error 37 us; skew 500 ppm
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056165, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:45.385128 29054 init.cc:377] Logged in from keytab as oryx/127.19.74.195@KRBTEST.COM (short username oryx)
I20260806 22:42:45.385576 29054 webserver.cc:460] Webserver started at http://127.19.74.195:40495/ using document root <none> and password file <none>
I20260806 22:42:45.385816 29054 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:45.385892 29054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:45.387259 29054 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:45.387833 29070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.388005 29054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:45.388100 29054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "6aa63c489787498993abd12387ffb667"
format_stamp: "Formatted at 2026-08-06 22:42:44 on dist-test-slave-pgkr"
server_key: "250e14c3a6c1691de7fe3042c5c81412"
server_key_iv: "9bb3073e5da1d81a04af8cff0307d560"
server_key_version: "encryptionkey@0"
I20260806 22:42:45.388417 29054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:45.396955 29054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:45.399101 29054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:45.399199 29054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:45.399348 29054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:45.399559 29054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:45.399585 29054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.399603 29054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:45.399626 29054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:45.405269 29054 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:45443
I20260806 22:42:45.405295 29184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:45443 every 8 connection(s)
I20260806 22:42:45.405648 29054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.412254 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.406306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:37789)
0806 22:42:45.406472 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.406509 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.407034 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.407289 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.407292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.407838 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.408010 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.408014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.408647 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.408647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.409366 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.409368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.409666 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.409667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.409755 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.410124 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.410132 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.412092 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
W20260806 22:42:45.412254 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.406327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:37789 (local address 127.19.74.254:32953)
0806 22:42:45.406372 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.406374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.406591 (+ 217us) server_negotiation.cc:407] Connection header received
0806 22:42:45.407138 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.407139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.407152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.407214 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.408084 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.408565 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.409432 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.409595 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.409647 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.412180 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:37789: BlockingRecv error: recv got EOF from 127.19.74.195:37789 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:42:45.412489 29185 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:32953 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.74.254:32953: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
I20260806 22:42:45.415516 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 29054
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.416891 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.412733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:38117)
0806 22:42:45.412785 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.412795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.412834 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.413042 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.413043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.413157 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.413197 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.413198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.413760 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.413760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.414265 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.414266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.414550 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.414551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.414593 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.415026 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.415031 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.416851 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:42:45.417032 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.412887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:38117 (local address 127.19.74.254:32953)
0806 22:42:45.412938 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.412939 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.412953 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:42:45.412973 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.412973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.412981 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.413028 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.413257 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.413699 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.414327 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.414491 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.414538 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.416956 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:38117: BlockingRecv error: recv got EOF from 127.19.74.195:38117 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:45 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:45.421705 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.417351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:58365)
0806 22:42:45.417405 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.417417 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:45.417459 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:45.417601 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:45.417602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:45.417764 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:45.417793 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.417794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.418352 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.418353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.419110 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:45.419113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.419462 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.419463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:45.419523 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.419846 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:45.419851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:45.421632 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:42:45.421756 28918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:45.417430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:58365 (local address 127.19.74.254:32953)
0806 22:42:45.417466 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.417467 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.417484 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:42:45.417504 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.417505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.417513 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.417589 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.417869 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.418274 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.419206 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.419388 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.419438 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.421694 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:58365: BlockingRecv error: recv got EOF from 127.19.74.195:58365 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:42:45.421967 29185 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:42:45.423171 28920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.415925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.19.74.254:32953)
0806 22:42:45.415984 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:45.415986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:45.416173 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:42:45.416241 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:45.416242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:45.416257 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:45.416315 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:45.417042 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.417659 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.418514 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:45.418739 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:45.418789 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:45.420123 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:45.420127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:45.420128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:45.420138 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:45.421932 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:45.422285 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:45.422288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:45.422289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:45.422380 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:45.422533 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:45.422533 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:45.422533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:45.422776 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:45.422882 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:45.422966 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 22:42:45.423041 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 22:42:46 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:46 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:46.163049 29189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:46.158297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:57683)
0806 22:42:46.158442 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:46.158453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:46.158557 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:46.158762 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:46.158763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:46.159000 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:46.159066 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:46.159069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.159771 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.159772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:46.160311 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:46.160312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.160643 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.160644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:46.160684 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:46.161104 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:46.161108 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:46.162981 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260806 22:42:46.163123 29190 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:46.158377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:57683 (local address 127.19.74.254:32953)
0806 22:42:46.158508 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:46.158510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:46.158534 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:42:46.158653 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:46.158654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:46.158668 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:46.158738 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:46.159148 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.159699 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.160379 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.160570 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.160625 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:46.163068 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:57683: BlockingRecv error: recv got EOF from 127.19.74.193:57683 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
Aug 06 22:42:46 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:46 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:46.305562 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:46.300393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:50325)
0806 22:42:46.300570 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:46.300584 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:46.300687 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:46.300885 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:46.300886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:46.301113 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:46.301167 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:46.301169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.301971 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.301972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:46.302578 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:46.302580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.303032 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.303033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:46.303107 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:46.303576 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:46.303581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:46.305495 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
W20260806 22:42:46.305719 29190 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:46.300485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:50325 (local address 127.19.74.254:32953)
0806 22:42:46.300570 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:46.300573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:46.300667 (+ 94us) server_negotiation.cc:407] Connection header received
0806 22:42:46.300781 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:46.300782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:46.300804 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:46.300879 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:46.301262 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.301848 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.302727 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.302928 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.303030 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:46.305616 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:50325: BlockingRecv error: recv got EOF from 127.19.74.194:50325 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 22:42:46 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:46 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:46.427120 29192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:46.422369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:34721)
0806 22:42:46.422529 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:46.422574 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:46.422681 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:46.422849 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:46.422849 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:46.423025 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:46.423074 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:46.423075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.423743 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.423744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:46.424293 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:46.424295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.424654 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.424655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:46.424703 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:46.425192 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:46.425197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:46.427047 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":204,"threads_started":1}
W20260806 22:42:46.427369 29190 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:46.422478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:34721 (local address 127.19.74.254:32953)
0806 22:42:46.422560 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:46.422564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:46.422647 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:42:46.422753 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:46.422754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:46.422767 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:46.422831 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:46.423152 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.423664 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.424378 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:46.424578 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:46.424643 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:46.427286 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:34721: BlockingRecv error: recv got EOF from 127.19.74.195:34721 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 22:42:47 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:47 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:47.168352 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:47.163567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:53159)
0806 22:42:47.163719 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:47.163734 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:47.163806 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:47.163925 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:47.163926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:47.164087 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:47.164161 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:47.164163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.164763 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.164764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:47.165336 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:47.165338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.165652 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.165653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:47.165693 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:47.166175 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:47.166180 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:47.168293 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
W20260806 22:42:47.168437 29194 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:47.163622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:53159 (local address 127.19.74.254:32953)
0806 22:42:47.163772 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:47.163773 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:47.163781 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:47.163849 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:47.163849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:47.163859 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:47.163911 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:47.164223 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.164699 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.165405 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.165590 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.165640 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:47.168366 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:53159: BlockingRecv error: recv got EOF from 127.19.74.193:53159 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
Aug 06 22:42:47 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:47 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:47.310506 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:47.306049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:45187)
0806 22:42:47.306214 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:47.306224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:47.306283 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:47.306409 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:47.306410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:47.306565 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:47.306632 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:47.306633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.307214 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.307215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:47.307727 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:47.307728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.308032 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.308033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:47.308103 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:47.308718 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:47.308723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:47.310450 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
W20260806 22:42:47.310681 29194 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:47.306100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:45187 (local address 127.19.74.254:32953)
0806 22:42:47.306179 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:47.306181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:47.306285 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:42:47.306338 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:47.306339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:47.306350 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:47.306390 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:47.306693 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.307152 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.307795 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.307971 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.308020 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:47.310560 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:45187: BlockingRecv error: recv got EOF from 127.19.74.194:45187 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:42:47 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:47 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:47.432305 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:47.427645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:47379)
0806 22:42:47.427813 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:47.427826 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:47.427891 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:47.428038 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:47.428038 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:47.428219 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:47.428270 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:47.428271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.428909 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.428910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:47.429429 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:47.429431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.429743 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.429743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:47.429781 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:47.430332 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:47.430336 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:47.432219 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
W20260806 22:42:47.432447 29194 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:47.427783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:47379 (local address 127.19.74.254:32953)
0806 22:42:47.427855 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:47.427857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:47.427868 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:47.427952 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:47.427953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:47.427965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:47.428015 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:47.428334 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.428845 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.429510 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:47.429681 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:47.429729 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:47.432362 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:47379: BlockingRecv error: recv got EOF from 127.19.74.195:47379 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 22:42:48 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:48 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:48.173570 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:48.168795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:60977)
0806 22:42:48.168962 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:48.168972 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:48.169022 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:48.169159 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:48.169159 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:48.169324 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:48.169373 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:48.169375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.170035 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.170036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:48.170571 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:48.170572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.170971 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.170972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:48.171015 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:48.171503 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:48.171513 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:48.173512 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
W20260806 22:42:48.173707 29198 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:48.168869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:60977 (local address 127.19.74.254:32953)
0806 22:42:48.169018 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:48.169020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:48.169029 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:48.169078 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:48.169079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:48.169089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:48.169139 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:48.169452 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.169969 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.170720 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.170908 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.170960 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:48.173602 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:60977: BlockingRecv error: recv got EOF from 127.19.74.193:60977 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
Aug 06 22:42:48 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:48 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:48.315989 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:48.311028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:55753)
0806 22:42:48.311203 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:48.311215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:48.311311 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:48.311472 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:48.311474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:48.311702 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:48.311751 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:48.311753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.312533 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.312535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:48.313054 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:48.313056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.313427 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.313428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:48.313496 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:48.313927 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:48.313932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:48.315858 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
W20260806 22:42:48.316072 29198 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:48.311116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:55753 (local address 127.19.74.254:32953)
0806 22:42:48.311197 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:48.311199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:48.311291 (+ 92us) server_negotiation.cc:407] Connection header received
0806 22:42:48.311378 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:48.311379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:48.311398 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:48.311465 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:48.311839 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.312416 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.313163 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.313355 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.313430 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:48.315983 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:55753: BlockingRecv error: recv got EOF from 127.19.74.194:55753 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:42:48 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:48 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:48.437047 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:48.432755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:45409)
0806 22:42:48.432924 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:48.432934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:48.432991 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:48.433121 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:48.433122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:48.433290 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:48.433339 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:48.433340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.433975 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.433976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:48.434472 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:48.434473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.434831 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.434832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:48.434871 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:48.435327 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:48.435331 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:48.436994 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
W20260806 22:42:48.437180 29198 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:48.432829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:45409 (local address 127.19.74.254:32953)
0806 22:42:48.432875 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:48.432877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:48.432976 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:42:48.433046 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:48.433046 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:48.433059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:48.433098 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:48.433402 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.433914 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.434539 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:48.434770 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:48.434822 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:48.437092 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:45409: BlockingRecv error: recv got EOF from 127.19.74.195:45409 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 22:42:49 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:49 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:49.178800 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:49.174147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:34497)
0806 22:42:49.174293 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:49.174303 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:49.174350 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:49.174475 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:49.174475 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:49.174704 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:49.174755 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:49.174757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.175392 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.175392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:49.175906 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:49.175907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.176224 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.176225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:49.176265 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:49.176821 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:49.176827 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:49.178742 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
W20260806 22:42:49.178961 29202 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:49.174210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:34497 (local address 127.19.74.254:32953)
0806 22:42:49.174351 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:49.174353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:49.174361 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:49.174387 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:49.174388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:49.174398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:49.174459 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:49.174827 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.175326 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.175975 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.176163 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.176213 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:49.178870 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:34497: BlockingRecv error: recv got EOF from 127.19.74.193:34497 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
Aug 06 22:42:49 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:49 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:49.321254 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:49.316458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:33775)
0806 22:42:49.316566 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:49.316574 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:49.316631 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:49.316757 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:49.316758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:49.316914 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:49.316963 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:49.316964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.317904 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.317906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:49.318446 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:49.318448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.318860 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.318860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:49.318905 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:49.319348 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:49.319353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:49.321158 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
W20260806 22:42:49.321297 29202 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:49.316528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:33775 (local address 127.19.74.254:32953)
0806 22:42:49.316583 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:49.316585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:49.316622 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:42:49.316672 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:49.316672 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:49.316684 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:49.316734 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:49.317017 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.317798 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.318516 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.318790 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.318844 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:49.321236 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:33775: BlockingRecv error: recv got EOF from 127.19.74.194:33775 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 22:42:49 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:49 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:49.441745 29204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:49.437471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:56559)
0806 22:42:49.437637 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:49.437647 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:49.437701 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:49.437829 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:49.437830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:49.437992 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:49.438043 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:49.438044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.438695 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.438696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:49.439207 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:49.439209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.439503 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.439503 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:49.439540 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:49.439968 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:49.439972 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:49.441690 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
W20260806 22:42:49.441883 29202 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:49.437550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:56559 (local address 127.19.74.254:32953)
0806 22:42:49.437643 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:49.437645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:49.437697 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:42:49.437745 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:49.437746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:49.437757 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:49.437809 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:49.438119 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.438625 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.439272 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:49.439441 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:49.439489 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:49.441800 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:56559: BlockingRecv error: recv got EOF from 127.19.74.195:56559 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 22:42:50 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:50 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:50.183818 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:50.179229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:33725)
0806 22:42:50.179383 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:50.179394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:50.179449 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:50.179599 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:50.179600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:50.179786 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:50.179838 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:50.179839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.180476 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.180476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:50.181111 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:50.181114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.181453 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.181454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:50.181493 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:50.181967 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:50.181972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:50.183758 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
W20260806 22:42:50.183965 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:50.179300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:33725 (local address 127.19.74.254:32953)
0806 22:42:50.179402 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:50.179404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:50.179438 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:42:50.179514 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:50.179515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:50.179525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:50.179581 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:50.179901 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.180413 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.181184 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.181385 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.181441 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:50.183873 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:33725: BlockingRecv error: recv got EOF from 127.19.74.193:33725 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
Aug 06 22:42:50 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:50 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:50.332037 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:50.321730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:43437)
0806 22:42:50.321888 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:50.321897 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:50.321945 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:50.322136 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:50.322137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:50.322301 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:50.322351 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:50.322353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.323414 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.323416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:50.324125 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:50.324127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.324478 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.324478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:50.324520 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:50.330014 (+ 5494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:50.330019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:50.331979 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
W20260806 22:42:50.332167 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:50.321858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:43437 (local address 127.19.74.254:32953)
0806 22:42:50.321928 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:50.321935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:50.321944 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:50.322030 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:50.322031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:50.322041 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:50.322089 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:50.322467 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.323298 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.324203 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.324405 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.324462 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:50.332084 (+ 7622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:43437: BlockingRecv error: recv got EOF from 127.19.74.194:43437 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:42:50 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:50 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:50.446484 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:50.442205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:55381)
0806 22:42:50.442365 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:50.442375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:50.442445 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:50.442577 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:50.442577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:50.442764 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:50.442813 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:50.442815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.443428 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.443429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:50.443942 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:50.443944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.444261 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.444262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:50.444302 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:50.444727 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:50.444731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:50.446427 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
W20260806 22:42:50.446668 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:50.442274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:55381 (local address 127.19.74.254:32953)
0806 22:42:50.442362 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:50.442364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:50.442433 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:42:50.442497 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:50.442498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:50.442517 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:50.442571 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:50.442873 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.443359 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.444011 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:50.444194 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:50.444247 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:50.446532 (+ 2285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:55381: BlockingRecv error: recv got EOF from 127.19.74.195:55381 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 22:42:51 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:51 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:51.189101 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:51.184264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:54299)
0806 22:42:51.184379 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:51.184387 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:51.184428 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:51.184620 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:51.184621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:51.184785 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:51.184836 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:51.184837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.185588 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.185589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:51.186266 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:51.186268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.186598 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.186599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:51.186684 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:51.187142 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:51.187146 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:51.189003 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
W20260806 22:42:51.189172 29210 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:51.184341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:54299 (local address 127.19.74.254:32953)
0806 22:42:51.184486 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:51.184489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:51.184496 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:51.184519 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:51.184519 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:51.184529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:51.184600 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:51.184900 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.185486 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.186332 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.186531 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.186588 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:51.189100 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:54299: BlockingRecv error: recv got EOF from 127.19.74.193:54299 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
Aug 06 22:42:51 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:51 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:42:51.338328 29210 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:51.332575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:52633 (local address 127.19.74.254:32953)
0806 22:42:51.332642 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:51.332644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:51.332762 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:42:51.332835 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:51.332836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:51.332851 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:51.332935 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:51.333283 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.334027 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.335038 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.335332 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.335373 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:51.338260 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:52633: BlockingRecv error: recv got EOF from 127.19.74.194:52633 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:42:51.338487 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:51.332510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:52633)
0806 22:42:51.332695 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:51.332705 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:51.332768 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:51.332941 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:51.332942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:51.333151 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:51.333210 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:51.333212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.334123 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.334124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:51.334940 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:51.334942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.335457 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.335457 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:51.335512 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:51.336003 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:51.336008 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:51.338421 (+ 2413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
Aug 06 22:42:51 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:51 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:51.451376 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:51.447074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:51623)
0806 22:42:51.447220 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:51.447231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:51.447298 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:51.447408 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:51.447409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:51.447572 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:51.447621 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:51.447622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.448240 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.448241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:51.448744 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:51.448746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.449065 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.449066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:51.449111 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:51.449537 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:51.449541 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:51.451318 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
W20260806 22:42:51.451516 29210 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:51.447162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:51623 (local address 127.19.74.254:32953)
0806 22:42:51.447210 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:51.447216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:51.447282 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:42:51.447343 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:51.447343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:51.447354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:51.447403 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:51.447678 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.448169 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.448810 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:51.448999 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:51.449052 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:51.451406 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:51623: BlockingRecv error: recv got EOF from 127.19.74.195:51623 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 22:42:52 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:52 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:52.193996 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:52.189551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:50671)
0806 22:42:52.189781 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:52.189791 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:52.189846 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:52.189958 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:52.189958 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:52.190112 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:52.190162 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:52.190164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.190826 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.190826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:52.191323 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:52.191324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.191666 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.191667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:52.191709 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:52.192161 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:52.192165 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:52.193897 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":175,"threads_started":1}
W20260806 22:42:52.194060 29214 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:52.189620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:50671 (local address 127.19.74.254:32953)
0806 22:42:52.189751 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:52.189753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:52.189831 (+ 78us) server_negotiation.cc:407] Connection header received
0806 22:42:52.189886 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:52.189886 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:52.189897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:52.189952 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:52.190216 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.190770 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.191417 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.191613 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.191664 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:52.193990 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:50671: BlockingRecv error: recv got EOF from 127.19.74.193:50671 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
Aug 06 22:42:52 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:52 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:52.343250 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:52.338908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:44181)
0806 22:42:52.339079 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:52.339090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:52.339152 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:52.339272 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:52.339273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:52.339434 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:52.339484 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:52.339486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.340110 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.340111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:52.340633 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:52.340635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.340948 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.340949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:52.340992 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:52.341420 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:52.341424 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:52.343182 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
W20260806 22:42:52.343348 29214 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:52.338971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:44181 (local address 127.19.74.254:32953)
0806 22:42:52.339036 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:52.339039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:52.339153 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:42:52.339205 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:52.339206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:52.339218 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:52.339266 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:52.339537 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.340040 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.340703 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.340884 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.340935 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:52.343279 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:44181: BlockingRecv error: recv got EOF from 127.19.74.194:44181 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:42:52 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:52 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:52.456465 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:52.451823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:58921)
0806 22:42:52.451957 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:52.451972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:52.452032 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:52.452162 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:52.452163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:52.452350 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:52.452400 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:52.452401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.453126 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.453126 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:52.453699 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:52.453701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.454032 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.454033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:52.454077 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:52.454524 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:52.454529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:52.456372 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
W20260806 22:42:52.456631 29214 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:52.451921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:58921 (local address 127.19.74.254:32953)
0806 22:42:52.451978 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:52.451980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:52.452035 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:42:52.452076 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:52.452076 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:52.452089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:52.452145 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:52.452546 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.453055 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.453779 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:52.453965 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:52.454019 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:52.456487 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:58921: BlockingRecv error: recv got EOF from 127.19.74.195:58921 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 22:42:53 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:53 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:53.199266 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:53.194564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:37705)
0806 22:42:53.194803 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:53.194814 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:53.194863 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:53.194991 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:53.194992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:53.195155 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:53.195206 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:53.195207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.195839 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.195840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:53.196345 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:53.196346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.196664 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.196665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:53.196705 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:53.197222 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:53.197227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:53.199159 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":168,"threads_started":1}
W20260806 22:42:53.199338 29218 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:53.194761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:37705 (local address 127.19.74.254:32953)
0806 22:42:53.194868 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:53.194870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:53.194880 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:53.194904 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:53.194904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:53.194914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:53.194972 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:53.195279 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.195775 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.196415 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.196602 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.196651 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:53.199255 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:37705: BlockingRecv error: recv got EOF from 127.19.74.193:37705 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":50,"threads_started":1}
Aug 06 22:42:53 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:53 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:53.348483 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:53.343931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:37725)
0806 22:42:53.344067 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:53.344076 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:53.344136 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:53.344245 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:53.344246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:53.344397 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:53.344464 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:53.344465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.345211 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.345211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:53.345742 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:53.345744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.346067 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.346068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:53.346112 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:53.346569 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:53.346573 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:53.348427 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260806 22:42:53.348592 29218 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:53.343988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:37725 (local address 127.19.74.254:32953)
0806 22:42:53.344057 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:53.344059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:53.344138 (+ 79us) server_negotiation.cc:407] Connection header received
0806 22:42:53.344181 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:53.344182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:53.344191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:53.344243 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:53.344527 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.345145 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.345813 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.346005 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.346053 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:53.348520 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:37725: BlockingRecv error: recv got EOF from 127.19.74.194:37725 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 22:42:53 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:53 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:53.461887 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:53.457026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:40699)
0806 22:42:53.457178 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:53.457187 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:53.457244 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:53.457374 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:53.457374 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:53.457536 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:53.457586 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:53.457587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.458432 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.458433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:53.459033 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:53.459036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.459362 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.459363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:53.459406 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:53.459903 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:53.459909 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:53.461790 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
W20260806 22:42:53.461943 29218 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:53.457094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:40699 (local address 127.19.74.254:32953)
0806 22:42:53.457147 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:53.457149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:53.457244 (+ 95us) server_negotiation.cc:407] Connection header received
0806 22:42:53.457299 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:53.457300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:53.457313 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:53.457353 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:53.457645 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.458349 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.459112 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:53.459293 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:53.459346 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:53.461876 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:40699: BlockingRecv error: recv got EOF from 127.19.74.195:40699 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 22:42:54 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:54 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:54.204460 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:54.199772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:56125)
0806 22:42:54.199961 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:54.199974 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:54.200061 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:54.200227 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:54.200227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:54.200396 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:54.200449 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:54.200451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.201070 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.201071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:54.201590 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:54.201592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.201905 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.201906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:54.201947 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.202437 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:54.202442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:54.204402 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
W20260806 22:42:54.204559 29222 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:54.199829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:56125 (local address 127.19.74.254:32953)
0806 22:42:54.199971 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:54.199973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:54.200043 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:42:54.200133 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:54.200134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:54.200151 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:54.200206 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:54.200510 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.201007 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.201666 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.201844 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.201892 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.204484 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:56125: BlockingRecv error: recv got EOF from 127.19.74.193:56125 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
Aug 06 22:42:54 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:54 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:54.354075 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:54.349011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:35869)
0806 22:42:54.349294 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:54.349306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:54.349396 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:54.349549 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:54.349551 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:54.349742 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:54.349791 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:54.349793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.350486 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.350487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:54.351058 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:54.351059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.351483 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.351484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:54.351547 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.352016 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:54.352020 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:54.354005 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":103,"threads_started":1}
W20260806 22:42:54.354197 29222 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:54.349112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:35869 (local address 127.19.74.254:32953)
0806 22:42:54.349289 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:54.349292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:54.349378 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:42:54.349456 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:54.349457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:54.349477 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:54.349542 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:54.349874 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.350389 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.351153 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.351397 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.351472 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.354128 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:35869: BlockingRecv error: recv got EOF from 127.19.74.194:35869 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 06 22:42:54 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:54 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:54.466898 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:54.462395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:52601)
0806 22:42:54.462542 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:54.462553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:54.462642 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:54.462774 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:54.462774 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:54.462947 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:54.462996 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:54.462997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.463634 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.463635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:54.464138 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:54.464140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.464440 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.464440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:54.464478 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.464937 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:54.464942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:54.466833 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
W20260806 22:42:54.466961 29222 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:54.462453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:52601 (local address 127.19.74.254:32953)
0806 22:42:54.462527 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:54.462530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:54.462643 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:42:54.462691 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:54.462692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:54.462705 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:54.462756 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:54.463053 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.463567 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.464201 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.464370 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.464427 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.466914 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:52601: BlockingRecv error: recv got EOF from 127.19.74.195:52601 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:42:55 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:55 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:55.209728 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.204960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:55205)
0806 22:42:55.205141 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.205152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.205217 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.205399 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.205401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.205587 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:55.205641 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.205643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.206290 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.206291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.206957 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.206960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.207309 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.207310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.207360 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.207922 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:55.207927 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:55.209667 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
W20260806 22:42:55.209838 29226 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:55.205011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:55205 (local address 127.19.74.254:32953)
0806 22:42:55.205145 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.205147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.205197 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:42:55.205272 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.205273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.205283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.205343 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:55.205710 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.206207 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.207024 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.207222 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.207286 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.209756 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:55205: BlockingRecv error: recv got EOF from 127.19.74.193:55205 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
Aug 06 22:42:55 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:55 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:55.365434 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.354667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:51443)
0806 22:42:55.354812 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.354824 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.354903 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.355069 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.355070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.355331 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:55.355426 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.355429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.356775 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.356777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.357770 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.357773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.358391 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.358392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.358435 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.363348 (+ 4913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:55.363353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:55.365309 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
W20260806 22:42:55.365460 29226 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:55.354707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:51443 (local address 127.19.74.254:32953)
0806 22:42:55.354776 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.354778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.354900 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:42:55.354976 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.354977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.354990 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.355048 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:55.355552 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.356663 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.357879 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.358245 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.358319 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.365412 (+ 7093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:51443: BlockingRecv error: recv got EOF from 127.19.74.194:51443 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 22:42:55 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:55 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:55.472492 29228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.467450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:44061)
0806 22:42:55.467748 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.467762 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.467834 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.467968 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.467969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.468148 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:55.468220 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.468222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.468915 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.468916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.469486 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.469487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.469950 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.469951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.470004 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.470413 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:55.470417 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:55.472380 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":76,"threads_started":1}
W20260806 22:42:55.472532 29226 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:55.467724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:44061 (local address 127.19.74.254:32953)
0806 22:42:55.467775 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.467776 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.467820 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:42:55.467881 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.467881 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.467893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.467952 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:55.468311 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.468827 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.469546 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.469887 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.469941 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.472464 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:44061: BlockingRecv error: recv got EOF from 127.19.74.195:44061 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 22:42:56 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:56 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:56.215301 29229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:56.210402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:52437)
0806 22:42:56.210566 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:56.210580 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:56.210680 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:56.210818 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:56.210819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:56.210993 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:56.211047 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:56.211049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.211694 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.211695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:56.212314 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:56.212316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.212649 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.212650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:56.212690 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:56.213167 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:56.213171 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:56.215188 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
W20260806 22:42:56.215376 29230 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:56.210468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:52437 (local address 127.19.74.254:32953)
0806 22:42:56.210649 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:56.210651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:56.210661 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:56.210730 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:56.210731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:56.210742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:56.210800 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:56.211122 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.211628 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.212392 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.212578 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.212632 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:56.215296 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:52437: BlockingRecv error: recv got EOF from 127.19.74.193:52437 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
Aug 06 22:42:56 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:56 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:56.370668 29231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:56.365917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:45173)
0806 22:42:56.366149 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:56.366159 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:56.366249 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:56.366474 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:56.366476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:56.366740 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:56.366795 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:56.366797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.367444 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.367445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:56.367964 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:56.367965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.368316 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.368316 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:56.368356 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:56.368779 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:56.368784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:56.370539 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":102,"threads_started":1}
W20260806 22:42:56.370723 29230 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:56.366035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:45173 (local address 127.19.74.254:32953)
0806 22:42:56.366128 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:56.366130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:56.366229 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:42:56.366312 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:56.366313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:56.366345 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:56.366422 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:56.366871 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.367364 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.368030 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.368229 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.368304 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:56.370654 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:45173: BlockingRecv error: recv got EOF from 127.19.74.194:45173 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 22:42:56 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:56 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:56.477706 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:56.473052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:43953)
0806 22:42:56.473191 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:56.473205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:56.473279 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:56.473417 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:56.473418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:56.473592 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:56.473652 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:56.473654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.474554 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.474555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:56.475125 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:56.475127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.475436 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.475437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:56.475476 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:56.475901 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:56.475905 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:56.477634 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
W20260806 22:42:56.477814 29230 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:56.473165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:43953 (local address 127.19.74.254:32953)
0806 22:42:56.473235 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:56.473237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:56.473265 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:42:56.473339 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:56.473340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:56.473352 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:56.473399 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:56.473715 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.474464 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.475192 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:56.475370 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:56.475422 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:56.477723 (+ 2301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:43953: BlockingRecv error: recv got EOF from 127.19.74.195:43953 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:42:57 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:57 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:57.220865 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:57.216005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:36903)
0806 22:42:57.216143 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:57.216156 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:57.216225 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:57.216418 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:57.216419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:57.216631 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:57.216691 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:57.216693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.217333 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.217334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:57.217904 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:57.217905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.218247 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.218248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:57.218304 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:57.218945 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:57.218951 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:57.220800 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
W20260806 22:42:57.220943 29234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:57.216070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:36903 (local address 127.19.74.254:32953)
0806 22:42:57.216206 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:57.216208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:57.216218 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:57.216288 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:57.216288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:57.216307 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:57.216395 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:57.216760 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.217258 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.217979 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.218160 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.218210 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:57.220888 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:36903: BlockingRecv error: recv got EOF from 127.19.74.193:36903 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
Aug 06 22:42:57 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:57 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:57.376027 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:57.371135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:57455)
0806 22:42:57.371337 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:57.371348 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:57.371416 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:57.371554 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:57.371555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:57.371791 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:57.371854 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:57.371856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.372579 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.372581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:57.373144 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:57.373146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.373536 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.373537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:57.373604 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:57.374075 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:57.374079 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:57.375910 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":125,"threads_started":1}
W20260806 22:42:57.376055 29234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:57.371251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:57455 (local address 127.19.74.254:32953)
0806 22:42:57.371323 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:57.371326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:57.371395 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:42:57.371472 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:57.371473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:57.371486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:57.371541 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:57.371950 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.372493 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.373236 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.373445 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.373538 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:57.375985 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:57455: BlockingRecv error: recv got EOF from 127.19.74.194:57455 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 22:42:57 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:57 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:57.482311 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:57.478143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:54259)
0806 22:42:57.478301 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:57.478311 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:57.478383 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:57.478507 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:57.478508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:57.478709 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:57.478755 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:57.478757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.479367 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.479368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:57.479889 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:57.479890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.480198 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.480198 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:57.480236 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:57.480660 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:57.480664 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:57.482257 (+ 1593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
W20260806 22:42:57.482450 29234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:57.478215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:54259 (local address 127.19.74.254:32953)
0806 22:42:57.478264 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:57.478267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:57.478350 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:42:57.478438 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:57.478439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:57.478448 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:57.478489 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:57.478819 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.479303 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.479960 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:57.480135 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:57.480187 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:57.482361 (+ 2174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:54259: BlockingRecv error: recv got EOF from 127.19.74.195:54259 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 22:42:58 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:58 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:58.226048 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:58.221372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:33511)
0806 22:42:58.221560 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:58.221570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:58.221635 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:58.221764 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:58.221765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:58.221957 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:58.222032 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:58.222033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.222712 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.222713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:58.223252 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:58.223254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.223581 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.223581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:58.223624 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:58.224058 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:58.224062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:58.225979 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
W20260806 22:42:58.226125 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:58.221436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:33511 (local address 127.19.74.254:32953)
0806 22:42:58.221563 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:58.221565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:58.221631 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:42:58.221682 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:58.221682 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:58.221692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:58.221744 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:58.222103 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.222593 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.223331 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.223514 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.223564 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:58.226069 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:33511: BlockingRecv error: recv got EOF from 127.19.74.193:33511 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Aug 06 22:42:58 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:58 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:58.381269 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:58.376561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:53655)
0806 22:42:58.376730 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:58.376741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:58.376795 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:58.376921 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:58.376922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:58.377102 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:58.377150 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:58.377152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.377822 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.377822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:58.378327 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:58.378328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.378732 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.378733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:58.378774 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:58.379264 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:58.379269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:58.381211 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
W20260806 22:42:58.381400 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:58.376669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:53655 (local address 127.19.74.254:32953)
0806 22:42:58.376742 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:58.376744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:58.376791 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:42:58.376843 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:58.376843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:58.376853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:58.376906 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:58.377210 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.377759 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.378394 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.378596 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.378717 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:58.381290 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:53655: BlockingRecv error: recv got EOF from 127.19.74.194:53655 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 22:42:58 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:58 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:58.487159 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:58.482809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:38391)
0806 22:42:58.482955 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:58.482965 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:58.483023 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:58.483147 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:58.483148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:58.483330 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:58.483375 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:58.483377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.483975 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.483976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:58.484505 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:58.484506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.484815 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.484816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:58.484854 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:58.485342 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:58.485346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:58.487102 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
W20260806 22:42:58.487265 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:58.482877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:38391 (local address 127.19.74.254:32953)
0806 22:42:58.482957 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:58.482959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:58.483008 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:42:58.483066 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:58.483067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:58.483079 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:58.483139 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:58.483438 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.483913 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.484573 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:58.484752 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:58.484803 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:58.487192 (+ 2389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:38391: BlockingRecv error: recv got EOF from 127.19.74.195:38391 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 22:42:59 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:59 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:59.231572 29241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:59.226738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:54551)
0806 22:42:59.226868 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:59.226879 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:59.226937 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:59.227119 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:59.227119 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:59.227339 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:59.227399 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:59.227402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.228222 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.228223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:59.228732 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:59.228734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.229098 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.229099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:59.229139 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:59.229572 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:59.229576 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:59.231468 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
W20260806 22:42:59.231626 29242 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:59.226818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:54551 (local address 127.19.74.254:32953)
0806 22:42:59.226955 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:59.226957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:59.226967 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:59.227009 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:59.227009 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:59.227021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:59.227088 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:59.227475 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.228147 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.228826 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.229018 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.229081 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:59.231553 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:54551: BlockingRecv error: recv got EOF from 127.19.74.193:54551 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
Aug 06 22:42:59 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:59 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:59.386046 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:59.381747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:45967)
0806 22:42:59.381882 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:59.381894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:59.381961 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:59.382086 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:59.382087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:59.382267 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:59.382319 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:59.382321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.382958 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.382958 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:59.383474 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:59.383476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.383797 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.383797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:59.383837 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:59.384272 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:59.384276 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:59.385987 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
W20260806 22:42:59.386165 29242 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:59.381826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:45967 (local address 127.19.74.254:32953)
0806 22:42:59.381908 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:59.381910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:59.381946 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:42:59.382018 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:59.382019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:59.382029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:59.382071 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:59.382384 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.382890 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.383544 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.383730 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.383786 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:59.386082 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:45967: BlockingRecv error: recv got EOF from 127.19.74.194:45967 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:42:59 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:42:59 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.195@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:42:59.492488 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:59.487733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.195:53251)
0806 22:42:59.487875 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:59.487885 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:59.487951 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:59.488078 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:59.488078 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:59.488255 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:59.488303 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:59.488304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.488933 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.488934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:59.489467 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:59.489468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.489804 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.489805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:59.489869 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:59.490300 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:59.490304 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:59.492391 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
W20260806 22:42:59.492539 29242 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:59.487788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:53251 (local address 127.19.74.254:32953)
0806 22:42:59.487852 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:59.487854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:59.487951 (+ 97us) server_negotiation.cc:407] Connection header received
0806 22:42:59.487995 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:59.487995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:59.488007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:59.488056 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:59.488363 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.488867 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.489536 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:59.489717 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:59.489781 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:59.492474 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.195:53251: BlockingRecv error: recv got EOF from 127.19.74.195:53251 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:43:00 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:43:00 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.193@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:43:00.236429 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.232063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.193:50647)
0806 22:43:00.232203 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:00.232213 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:00.232272 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:00.232408 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:00.232408 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:00.232573 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:00.232628 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.232630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.233250 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.233251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.233779 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.233781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.234095 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.234096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.234138 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.234594 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:00.234599 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:00.236302 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
W20260806 22:43:00.236471 29246 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:43:00.232152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:50647 (local address 127.19.74.254:32953)
0806 22:43:00.232267 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:00.232269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:00.232296 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:43:00.232336 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:00.232337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:00.232348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:00.232402 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:00.232693 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.233184 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.233846 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.234027 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.234076 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.236394 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.193:50647: BlockingRecv error: recv got EOF from 127.19.74.193:50647 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":53,"threads_started":1}
Aug 06 22:43:00 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:43:00 dist-test-slave-pgkr krb5kdc[28167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.74.194@KRBTEST.COM for oryx/127.19.74.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:43:00.395948 29247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.386660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:32953 (local address 127.19.74.194:50499)
0806 22:43:00.386830 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:00.386843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:00.386946 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:00.387131 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:00.387133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:00.387332 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:00.387382 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.387384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.388151 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.388153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.388770 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.388772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.389179 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.389180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.389253 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.393831 (+ 4578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:00.393845 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:00.395829 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.74.254:32953: Server oryx/127.19.74.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
W20260806 22:43:00.395996 29246 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:43:00.386722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:50499 (local address 127.19.74.254:32953)
0806 22:43:00.386814 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:00.386817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:00.386924 (+ 107us) server_negotiation.cc:407] Connection header received
0806 22:43:00.387025 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:00.387026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:00.387044 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:00.387115 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:00.387471 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.388033 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.388863 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.389082 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.389165 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.395924 (+ 6759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.74.194:50499: BlockingRecv error: recv got EOF from 127.19.74.194:50499 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:43:00.423262 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 28765
I20260806 22:43:00.429347 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 28919
I20260806 22:43:00.434684 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 29054
I20260806 22:43:00.439489 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 28691
2026-08-06T22:43:00Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17518 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T22:43:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:00Z Disabled control of system clock
I20260806 22:43:00.469198 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38059
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:33073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:43:00.550889 29251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:00.551069 29251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:00.551087 29251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:00.552296 29251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:00.552340 29251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:00.552352 29251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:00.552388 29251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:00.552414 29251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:00.553824 29251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38059
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:00.554075 29251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:00.554363 29251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:00.559422 29259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:00.559398 29256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:00.559403 29257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:00.559430 29251 server_base.cc:1034] running on GCE node
I20260806 22:43:00.559952 29251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:00.560242 29251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:00.561388 29251 hybrid_clock.cc:648] HybridClock initialized: now 1786056180561372 us; error 36 us; skew 500 ppm
I20260806 22:43:00.562510 29251 webserver.cc:460] Webserver started at http://127.19.74.254:34217/ using document root <none> and password file <none>
I20260806 22:43:00.562781 29251 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:00.562829 29251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:00.562979 29251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:00.564035 29251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "a8298fad1a544fe8bffa1b5246418285"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "5f45c3223164ab8cbc59f759356f051a"
server_key_iv: "a863554340e0c40fe0d6decfc28ff5d9"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.564383 29251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "a8298fad1a544fe8bffa1b5246418285"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "5f45c3223164ab8cbc59f759356f051a"
server_key_iv: "a863554340e0c40fe0d6decfc28ff5d9"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.565640 29251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:00.566321 29265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.566493 29251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:00.566641 29251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "a8298fad1a544fe8bffa1b5246418285"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "5f45c3223164ab8cbc59f759356f051a"
server_key_iv: "a863554340e0c40fe0d6decfc28ff5d9"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.566732 29251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:00.584534 29251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:00.584836 29251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:00.584986 29251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:00.588943 29251 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:38059
I20260806 22:43:00.588979 29318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:38059 every 8 connection(s)
I20260806 22:43:00.589314 29251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:00.590042 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:00.592741 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285: Bootstrap starting.
I20260806 22:43:00.593349 29319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:00.593588 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 29251
I20260806 22:43:00.593662 29319 log.cc:824] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:00.593704 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:43:00.593914 19755 external_mini_cluster.cc:1428] Setting key 756fe9081b4e81a69673dd731f452f30
I20260806 22:43:00.594828 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285: No bootstrap required, opened a new log
I20260806 22:43:00.596379 29319 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8298fad1a544fe8bffa1b5246418285" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38059 } }
I20260806 22:43:00.596532 29319 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:00.596580 29319 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8298fad1a544fe8bffa1b5246418285, State: Initialized, Role: FOLLOWER
I20260806 22:43:00.596737 29319 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8298fad1a544fe8bffa1b5246418285" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38059 } }
I20260806 22:43:00.596845 29319 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:00.596890 29319 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:00.596937 29319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:00.597460 29319 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8298fad1a544fe8bffa1b5246418285" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38059 } }
I20260806 22:43:00.597577 29319 leader_election.cc:302] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8298fad1a544fe8bffa1b5246418285; no voters:
I20260806 22:43:00.597786 29319 leader_election.cc:288] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:00.597812 29324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:00.597929 29324 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [term 1 LEADER]: Becoming Leader. State: Replica: a8298fad1a544fe8bffa1b5246418285, State: Running, Role: LEADER
I20260806 22:43:00.598074 29319 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:00.598085 29324 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8298fad1a544fe8bffa1b5246418285" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38059 } }
I20260806 22:43:00.598542 29324 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8298fad1a544fe8bffa1b5246418285. Latest consensus state: current_term: 1 leader_uuid: "a8298fad1a544fe8bffa1b5246418285" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8298fad1a544fe8bffa1b5246418285" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38059 } } }
I20260806 22:43:00.598661 29326 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8298fad1a544fe8bffa1b5246418285" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8298fad1a544fe8bffa1b5246418285" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38059 } } }
I20260806 22:43:00.598737 29324 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:00.598755 29326 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:00.599047 29333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:00.599757 29333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:00.600621 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.594461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50438 (local address 127.19.74.254:38059)
0806 22:43:00.594725 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:00.594733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:00.594758 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:43:00.595210 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:00.595218 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:00.595278 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:00.595428 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:00.595620 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.597170 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.598116 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.598702 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.598798 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.599756 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:00.599760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:00.599760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:00.599957 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:00.600005 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:00.600021 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:00.600077 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:00.600326 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:43:00.600401 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":69,"threads_started":1}
I20260806 22:43:00.601637 29333 catalog_manager.cc:1347] Generated new cluster ID: 7ea5750bc4f24a1f9083c1a6261b8642
I20260806 22:43:00.601691 29333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:43:00.615026 29333 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:00.615495 29333 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:00.618060 29333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8298fad1a544fe8bffa1b5246418285: Generated new TSK 0
I20260806 22:43:00.618216 29333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:43:00.622414 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38059
--builtin_ntp_servers=127.19.74.212:33073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:43:00.704658 29343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:00.704905 29343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:00.704941 29343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:00.706017 29343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:00.706077 29343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:00.706123 29343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:43:00.707399 29343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:00.707662 29343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:00.707955 29343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:00.712873 29348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:00.712858 29349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:00.712846 29351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:00.713373 29343 server_base.cc:1034] running on GCE node
I20260806 22:43:00.713649 29343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:00.713958 29343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:00.715168 29343 hybrid_clock.cc:648] HybridClock initialized: now 1786056180715136 us; error 46 us; skew 500 ppm
I20260806 22:43:00.716495 29343 webserver.cc:460] Webserver started at http://127.19.74.193:36735/ using document root <none> and password file <none>
I20260806 22:43:00.716763 29343 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:00.716817 29343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:00.716948 29343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:00.718217 29343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "bbf9e09030a540d78a97ae05c55bff14"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "189a11d3ac8970ed4fd2f920076936e2"
server_key_iv: "4d05ed4ac211a35099eb7fa76c212952"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.718629 29343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "bbf9e09030a540d78a97ae05c55bff14"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "189a11d3ac8970ed4fd2f920076936e2"
server_key_iv: "4d05ed4ac211a35099eb7fa76c212952"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.720276 29343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:00.721118 29357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.721323 29343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:00.721413 29343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "bbf9e09030a540d78a97ae05c55bff14"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "189a11d3ac8970ed4fd2f920076936e2"
server_key_iv: "4d05ed4ac211a35099eb7fa76c212952"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.721477 29343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:00.738405 29343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:00.738735 29343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:00.738838 29343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:00.738997 29343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:00.739331 29343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:00.739364 29343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.739420 29343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:00.739444 29343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.745272 29343 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:42709
I20260806 22:43:00.745352 29474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:42709 every 8 connection(s)
I20260806 22:43:00.745693 29343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:00.748081 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 29343
I20260806 22:43:00.748203 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:43:00.748397 19755 external_mini_cluster.cc:1428] Setting key 32b03bf986a35ac765f8d30a2d431cc8
I20260806 22:43:00.749465 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38059
--builtin_ntp_servers=127.19.74.212:33073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:43:00.751560 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.746398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:60905 (local address 127.19.74.254:38059)
0806 22:43:00.746445 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:00.746447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:00.746707 (+ 260us) server_negotiation.cc:407] Connection header received
0806 22:43:00.747277 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:00.747278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:00.747295 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:00.747360 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:00.748073 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.748754 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.750129 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.750299 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.750351 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.751267 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:00.751268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:00.751269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:00.751274 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:00.751290 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:00.751295 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:00.751321 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:00.751461 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:43:00.751484 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:43:00.751775 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.746359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38059 (local address 127.19.74.193:60905)
0806 22:43:00.746535 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:00.746567 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:00.747165 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:00.747426 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:00.747430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:00.747440 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:00.747966 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.747972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.748827 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.748828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.750050 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.750054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.750379 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.750380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.750527 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.751079 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:00.751080 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:00.751120 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:00.751151 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:00.751360 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:00.751367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:00.751376 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:43:00.751446 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:43:00.751546 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260806 22:43:00.752229 29475 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38059
I20260806 22:43:00.752343 29475 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:00.752545 29475 heartbeater.cc:511] Master 127.19.74.254:38059 requested a full tablet report, sending...
I20260806 22:43:00.753069 29282 ts_manager.cc:194] Registered new tserver with Master: bbf9e09030a540d78a97ae05c55bff14 (127.19.74.193:42709)
I20260806 22:43:00.753705 29282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.193:60905
W20260806 22:43:00.830226 29478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:00.830428 29478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:00.830451 29478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:00.831676 29478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:00.831719 29478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:00.831745 29478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:43:00.833135 29478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:00.833376 29478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:00.833680 29478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:00.838490 29486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:00.838517 29483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:00.838517 29484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:00.838644 29478 server_base.cc:1034] running on GCE node
I20260806 22:43:00.838836 29478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:00.839148 29478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:00.840312 29478 hybrid_clock.cc:648] HybridClock initialized: now 1786056180840300 us; error 28 us; skew 500 ppm
I20260806 22:43:00.841614 29478 webserver.cc:460] Webserver started at http://127.19.74.194:33521/ using document root <none> and password file <none>
I20260806 22:43:00.841862 29478 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:00.841931 29478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:00.842069 29478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:00.843130 29478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "dfb20ade1ee14ba199e2e9cbe84c701f"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "33315070b218b982b63287dd474ec4cc"
server_key_iv: "60c9161f2d0a7cd1ec55d62c2fadc5c9"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.843462 29478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "dfb20ade1ee14ba199e2e9cbe84c701f"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "33315070b218b982b63287dd474ec4cc"
server_key_iv: "60c9161f2d0a7cd1ec55d62c2fadc5c9"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.844671 29478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:00.845373 29492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.845552 29478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:00.845654 29478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "dfb20ade1ee14ba199e2e9cbe84c701f"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "33315070b218b982b63287dd474ec4cc"
server_key_iv: "60c9161f2d0a7cd1ec55d62c2fadc5c9"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.845736 29478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:00.856871 29478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:00.857164 29478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:00.857311 29478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:00.857528 29478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:00.857856 29478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:00.857909 29478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.857942 29478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:00.857968 29478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.863811 29478 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:38767
I20260806 22:43:00.863852 29607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:38767 every 8 connection(s)
I20260806 22:43:00.864161 29478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:43:00.865760 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 29478
I20260806 22:43:00.865859 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:00.866094 19755 external_mini_cluster.cc:1428] Setting key 191b7a5a983293a89c18adf76d64eee6
I20260806 22:43:00.867072 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38059
--builtin_ntp_servers=127.19.74.212:33073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:43:00.869563 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.864846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:44347 (local address 127.19.74.254:38059)
0806 22:43:00.864901 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:00.864903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:00.865137 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:43:00.865535 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:00.865536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:00.865545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:00.865587 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:00.866424 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.867011 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.868053 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.868258 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.868333 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.869261 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:00.869262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:00.869263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:00.869268 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:00.869284 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:00.869288 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:00.869342 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:00.869459 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:43:00.869486 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:43:00.869810 29610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.864822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38059 (local address 127.19.74.194:44347)
0806 22:43:00.864999 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:00.865036 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:00.865463 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:00.865664 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:00.865669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:00.865680 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:00.866355 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.866360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.867107 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.867108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.867945 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.867947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.868341 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.868342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.868474 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.869100 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:00.869100 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:00.869130 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:00.869152 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:00.869346 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:00.869351 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:00.869354 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:00.869458 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:43:00.869561 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260806 22:43:00.870342 29608 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38059
I20260806 22:43:00.870430 29608 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:00.870754 29608 heartbeater.cc:511] Master 127.19.74.254:38059 requested a full tablet report, sending...
I20260806 22:43:00.871155 29282 ts_manager.cc:194] Registered new tserver with Master: dfb20ade1ee14ba199e2e9cbe84c701f (127.19.74.194:38767)
I20260806 22:43:00.871547 29282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.194:44347
W20260806 22:43:00.949530 29611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:00.949760 29611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:00.949791 29611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:00.950970 29611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:00.951036 29611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:00.951077 29611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:43:00.952428 29611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:00.952687 29611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:00.952973 29611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:00.957773 29616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:00.957901 29611 server_base.cc:1034] running on GCE node
W20260806 22:43:00.957787 29617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:00.957814 29619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:00.958213 29611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:00.958503 29611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:00.959688 29611 hybrid_clock.cc:648] HybridClock initialized: now 1786056180959689 us; error 41 us; skew 500 ppm
I20260806 22:43:00.960927 29611 webserver.cc:460] Webserver started at http://127.19.74.195:46731/ using document root <none> and password file <none>
I20260806 22:43:00.961167 29611 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:00.961234 29611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:00.961369 29611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:00.962409 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "89ccbe0e8256463f97753c76ee0c7bce"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "2e5708d5c7593fa28cf14b8677d9f8fc"
server_key_iv: "fbda4aa5d89d05907f1f57daff2b2bfd"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.962852 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "89ccbe0e8256463f97753c76ee0c7bce"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "2e5708d5c7593fa28cf14b8677d9f8fc"
server_key_iv: "fbda4aa5d89d05907f1f57daff2b2bfd"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.964102 29611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:00.964779 29625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.964951 29611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:00.965055 29611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "89ccbe0e8256463f97753c76ee0c7bce"
format_stamp: "Formatted at 2026-08-06 22:43:00 on dist-test-slave-pgkr"
server_key: "2e5708d5c7593fa28cf14b8677d9f8fc"
server_key_iv: "fbda4aa5d89d05907f1f57daff2b2bfd"
server_key_version: "encryptionkey@0"
I20260806 22:43:00.965140 29611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:00.983799 29611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:00.984074 29611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:00.984241 29611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:00.984462 29611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:00.984782 29611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:00.984836 29611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.984880 29611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:00.984908 29611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:00.990396 29611 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:33829
I20260806 22:43:00.990473 29739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:33829 every 8 connection(s)
I20260806 22:43:00.990811 29611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:43:00.993257 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 29611
I20260806 22:43:00.993361 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:00.993580 19755 external_mini_cluster.cc:1428] Setting key 047d22ffed731588a6db61ac5df3d2d6
I20260806 22:43:00.996142 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.991505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:49633 (local address 127.19.74.254:38059)
0806 22:43:00.991552 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:00.991554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:00.991790 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:43:00.992340 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:00.992342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:00.992355 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:00.992402 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:00.993076 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.993742 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.994723 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.994925 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.995008 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.995852 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:00.995854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:00.995854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:00.995858 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:00.995873 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:00.995877 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:00.995919 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:00.996021 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:43:00.996045 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:43:00.996316 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:00.991481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38059 (local address 127.19.74.195:49633)
0806 22:43:00.991679 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:00.991708 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:00.992204 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:00.992486 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:00.992489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:00.992497 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:00.992981 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.992985 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.993830 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.993831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.994570 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:00.994572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:00.995019 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:00.995020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:00.995146 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:00.995686 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:00.995687 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:00.995715 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:00.995735 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:00.995930 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:00.995934 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:00.995937 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:00.996016 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:43:00.996114 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260806 22:43:00.996739 29740 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38059
I20260806 22:43:00.996819 29740 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:00.996948 29740 heartbeater.cc:511] Master 127.19.74.254:38059 requested a full tablet report, sending...
I20260806 22:43:00.997263 29282 ts_manager.cc:194] Registered new tserver with Master: 89ccbe0e8256463f97753c76ee0c7bce (127.19.74.195:33829)
I20260806 22:43:00.997555 29282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.195:49633
I20260806 22:43:01.005270 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:43:01.007102 29323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:43:01.006720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50440 (local address 127.19.74.254:38059)
0806 22:43:01.006776 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:01.006778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:01.006790 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:43:01.006862 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:01.006862 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:01.006870 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:01.006927 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:01.007032 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50440: BlockingRecv error: recv got EOF from 127.0.0.1:50440 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:43:01.007194 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 29343
I20260806 22:43:01.015231 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 29478
I20260806 22:43:01.021010 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 29611
I20260806 22:43:01.025817 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 29251
2026-08-06T22:43:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (585 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T22:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:01Z Disabled control of system clock
I20260806 22:43:01.055182 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34981
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:34981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:43:01.136754 29751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:01.136950 29751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:01.136970 29751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:01.137975 29751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:01.138006 29751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:01.138019 29751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:01.138031 29751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:01.138042 29751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:01.139351 29751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:34981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:01.139547 29751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:01.139796 29751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:01.144706 29760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:01.144702 29757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:01.144735 29758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:01.144942 29751 server_base.cc:1034] running on GCE node
I20260806 22:43:01.145190 29751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:01.145473 29751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:01.146664 29751 hybrid_clock.cc:648] HybridClock initialized: now 1786056181146648 us; error 34 us; skew 500 ppm
I20260806 22:43:01.147858 29751 webserver.cc:460] Webserver started at http://127.19.74.254:42661/ using document root <none> and password file <none>
I20260806 22:43:01.148116 29751 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:01.148185 29751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:01.148326 29751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:01.149370 29751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "593ec64311782a9518b2055464401996"
server_key_iv: "77ccee1903622a0b8a69de69486325f1"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.149739 29751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "593ec64311782a9518b2055464401996"
server_key_iv: "77ccee1903622a0b8a69de69486325f1"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.151139 29751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:01.151836 29766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.152012 29751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:01.152141 29751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "593ec64311782a9518b2055464401996"
server_key_iv: "77ccee1903622a0b8a69de69486325f1"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.152216 29751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:01.155650 29751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:01.155912 29751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:01.156062 29751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:01.160156 29751 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:34981
I20260806 22:43:01.160187 29818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:34981 every 8 connection(s)
I20260806 22:43:01.160516 29751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:01.161191 29819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:01.164069 29819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d: Bootstrap starting.
I20260806 22:43:01.164649 29819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:01.164951 29819 log.cc:824] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:01.165612 29819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d: No bootstrap required, opened a new log
I20260806 22:43:01.167268 29819 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34981 } }
I20260806 22:43:01.167398 29819 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:01.167419 29819 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5e4a2f50064c2292fc6a3eb7c7bc1d, State: Initialized, Role: FOLLOWER
I20260806 22:43:01.167554 29819 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34981 } }
I20260806 22:43:01.167639 29819 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:01.167694 29819 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:01.167742 29819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:01.168268 29819 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34981 } }
I20260806 22:43:01.168411 29819 leader_election.cc:302] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec5e4a2f50064c2292fc6a3eb7c7bc1d; no voters:
I20260806 22:43:01.168596 29819 leader_election.cc:288] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:01.168658 29822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:01.168853 29819 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:01.168885 29822 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [term 1 LEADER]: Becoming Leader. State: Replica: ec5e4a2f50064c2292fc6a3eb7c7bc1d, State: Running, Role: LEADER
I20260806 22:43:01.168970 29822 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34981 } }
I20260806 22:43:01.169536 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 29751
I20260806 22:43:01.169631 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:43:01.169826 19755 external_mini_cluster.cc:1428] Setting key 7314ec693b5200bf32982f7e4e6a33bc
I20260806 22:43:01.169982 29823 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34981 } } }
I20260806 22:43:01.170071 29823 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:01.170159 29824 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5e4a2f50064c2292fc6a3eb7c7bc1d. Latest consensus state: current_term: 1 leader_uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e4a2f50064c2292fc6a3eb7c7bc1d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34981 } } }
I20260806 22:43:01.170238 29824 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:01.170672 29831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:01.171221 29831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:01.172505 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:01.170514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52556 (local address 127.19.74.254:34981)
0806 22:43:01.170913 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:01.170924 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:01.170967 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:43:01.171101 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:01.171108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:01.171162 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:01.171589 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:01.171990 (+ 401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:01.171993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:01.171993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:01.171996 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:01.172044 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:01.172056 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:01.172134 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:01.172207 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 22:43:01.172326 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":119,"threads_started":1}
I20260806 22:43:01.173272 29831 catalog_manager.cc:1347] Generated new cluster ID: d7925638f54e40849c5baf3a9ae42b9b
I20260806 22:43:01.173324 29831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:43:01.187984 29831 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:01.188160 29831 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:01.193647 29831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec5e4a2f50064c2292fc6a3eb7c7bc1d: Generated new TSK 0
I20260806 22:43:01.193851 29831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:43:01.197755 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:34981
--builtin_ntp_servers=127.19.74.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:43:01.285925 29843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:01.286147 29843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:01.286185 29843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:01.287369 29843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:01.287426 29843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:01.287470 29843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:43:01.288861 29843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:34981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:01.289121 29843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:01.289414 29843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:01.295434 29851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:01.295475 29848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:01.295434 29849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:01.295781 29843 server_base.cc:1034] running on GCE node
I20260806 22:43:01.296034 29843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:01.296329 29843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:01.297505 29843 hybrid_clock.cc:648] HybridClock initialized: now 1786056181297491 us; error 27 us; skew 500 ppm
I20260806 22:43:01.298856 29843 webserver.cc:460] Webserver started at http://127.19.74.193:41903/ using document root <none> and password file <none>
I20260806 22:43:01.299108 29843 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:01.299170 29843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:01.299293 29843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:01.300606 29843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "1f09185d63714c32bf57cb59bb52fb56"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "7b5cf9ffea7e48dac5e5cb7e9999912d"
server_key_iv: "d48a9e63da4e39cbdd76f1fbfe7b646c"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.300974 29843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "1f09185d63714c32bf57cb59bb52fb56"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "7b5cf9ffea7e48dac5e5cb7e9999912d"
server_key_iv: "d48a9e63da4e39cbdd76f1fbfe7b646c"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.302523 29843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:43:01.303508 29857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.303690 29843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:01.303789 29843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "1f09185d63714c32bf57cb59bb52fb56"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "7b5cf9ffea7e48dac5e5cb7e9999912d"
server_key_iv: "d48a9e63da4e39cbdd76f1fbfe7b646c"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.303872 29843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:01.307143 29843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:01.307394 29843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:01.307535 29843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:01.307749 29843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:01.308060 29843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:01.308101 29843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.308125 29843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:01.308164 29843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.313534 29843 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:41429
I20260806 22:43:01.313623 29972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:41429 every 8 connection(s)
I20260806 22:43:01.313916 29843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:01.315637 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 29843
I20260806 22:43:01.315750 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:43:01.315958 19755 external_mini_cluster.cc:1428] Setting key 5176d3d5c05462f0efcfe154b3b3bb07
I20260806 22:43:01.317067 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:01.314647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:36707 (local address 127.19.74.254:34981)
0806 22:43:01.314701 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:01.314703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:01.314919 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:43:01.315405 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:01.315406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:01.315414 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:01.315471 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:01.316815 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:01.316817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:01.316817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:01.316821 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:01.316840 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:01.316845 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:01.316914 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:01.316962 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 22:43:01.316977 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:43:01.317083 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:34981
--builtin_ntp_servers=127.19.74.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:43:01.317195 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:01.314603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:34981 (local address 127.19.74.193:36707)
0806 22:43:01.314805 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:01.314837 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:01.315300 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:01.315558 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:01.315561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:01.315569 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:01.316646 (+ 1077us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:01.316646 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:01.316684 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:01.316719 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:01.316910 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:01.316913 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:01.316915 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:43:01.316964 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:43:01.317010 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":116,"threads_started":1}
I20260806 22:43:01.317692 29973 heartbeater.cc:347] Connected to a master server at 127.19.74.254:34981
I20260806 22:43:01.317786 29973 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:01.317929 29973 heartbeater.cc:511] Master 127.19.74.254:34981 requested a full tablet report, sending...
I20260806 22:43:01.318387 29783 ts_manager.cc:194] Registered new tserver with Master: 1f09185d63714c32bf57cb59bb52fb56 (127.19.74.193:41429)
W20260806 22:43:01.399209 29976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:01.399410 29976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:01.399431 29976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:01.400452 29976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:01.400503 29976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:01.400528 29976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:43:01.401832 29976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:34981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:01.402007 29976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:01.402261 29976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:01.407317 29981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:01.407436 29984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:01.407437 29982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:01.407846 29976 server_base.cc:1034] running on GCE node
I20260806 22:43:01.408062 29976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:01.408298 29976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:01.409489 29976 hybrid_clock.cc:648] HybridClock initialized: now 1786056181409475 us; error 23 us; skew 500 ppm
I20260806 22:43:01.410816 29976 webserver.cc:460] Webserver started at http://127.19.74.194:36553/ using document root <none> and password file <none>
I20260806 22:43:01.411082 29976 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:01.411158 29976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:01.411305 29976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:01.412356 29976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "ba92a9d3c2644a2c9f6a57751708a168"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "43b0637c198b0d953284d49e629d9788"
server_key_iv: "c036c85e170301570f63c123b6a00d88"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.412717 29976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "ba92a9d3c2644a2c9f6a57751708a168"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "43b0637c198b0d953284d49e629d9788"
server_key_iv: "c036c85e170301570f63c123b6a00d88"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.413986 29976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:01.414690 29990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.414870 29976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:01.414969 29976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "ba92a9d3c2644a2c9f6a57751708a168"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "43b0637c198b0d953284d49e629d9788"
server_key_iv: "c036c85e170301570f63c123b6a00d88"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.415050 29976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:01.418347 29976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:01.418655 29976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:01.418834 29976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:01.419052 29976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:01.419355 29976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:01.419409 29976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.419446 29976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:01.419473 29976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.425150 29976 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:36301
I20260806 22:43:01.425185 30106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:36301 every 8 connection(s)
I20260806 22:43:01.425503 29976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:43:01.428369 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:01.426289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:42075 (local address 127.19.74.254:34981)
0806 22:43:01.426356 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:01.426358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:01.426562 (+ 204us) server_negotiation.cc:407] Connection header received
0806 22:43:01.427056 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:01.427057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:01.427061 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:01.427123 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:01.428160 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:01.428161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:01.428161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:01.428165 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:01.428179 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:01.428183 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:01.428226 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:01.428286 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:43:01.428298 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:43:01.428568 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:01.426259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:34981 (local address 127.19.74.194:42075)
0806 22:43:01.426438 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:01.426475 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:01.426970 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:01.427172 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:01.427175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:01.427182 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:01.428022 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:01.428022 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:01.428050 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:01.428077 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:01.428230 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:01.428232 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:01.428235 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:01.428282 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:43:01.428327 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260806 22:43:01.429066 30107 heartbeater.cc:347] Connected to a master server at 127.19.74.254:34981
I20260806 22:43:01.429150 30107 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:01.429265 30107 heartbeater.cc:511] Master 127.19.74.254:34981 requested a full tablet report, sending...
I20260806 22:43:01.429576 29783 ts_manager.cc:194] Registered new tserver with Master: ba92a9d3c2644a2c9f6a57751708a168 (127.19.74.194:36301)
I20260806 22:43:01.432753 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 29976
I20260806 22:43:01.432842 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:01.433007 19755 external_mini_cluster.cc:1428] Setting key 699a495633a127bf18aefeb448b7bda2
I20260806 22:43:01.433921 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:34981
--builtin_ntp_servers=127.19.74.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:43:01.516490 30110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:01.516687 30110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:01.516708 30110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:01.517885 30110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:01.517922 30110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:01.517946 30110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:43:01.519464 30110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:34981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:01.519797 30110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:01.520165 30110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:01.525167 30116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:01.525214 30115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:01.525183 30118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:01.525406 30110 server_base.cc:1034] running on GCE node
I20260806 22:43:01.525681 30110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:01.525949 30110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:01.527103 30110 hybrid_clock.cc:648] HybridClock initialized: now 1786056181527079 us; error 35 us; skew 500 ppm
I20260806 22:43:01.528373 30110 webserver.cc:460] Webserver started at http://127.19.74.195:37375/ using document root <none> and password file <none>
I20260806 22:43:01.528635 30110 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:01.528704 30110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:01.528859 30110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:01.529891 30110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "02d36af6904541008898ffe584f6c3ad"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "3031d660fed4bd9cab574995b57204b6"
server_key_iv: "e50d652a1bdedca46a20180e7035e5c0"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.530174 30110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "02d36af6904541008898ffe584f6c3ad"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "3031d660fed4bd9cab574995b57204b6"
server_key_iv: "e50d652a1bdedca46a20180e7035e5c0"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.531443 30110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:43:01.532123 30124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.532307 30110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:01.532379 30110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "02d36af6904541008898ffe584f6c3ad"
format_stamp: "Formatted at 2026-08-06 22:43:01 on dist-test-slave-pgkr"
server_key: "3031d660fed4bd9cab574995b57204b6"
server_key_iv: "e50d652a1bdedca46a20180e7035e5c0"
server_key_version: "encryptionkey@0"
I20260806 22:43:01.532429 30110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:01.535705 30110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:01.535917 30110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:01.536010 30110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:01.536167 30110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:01.536403 30110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:01.536432 30110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.536450 30110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:01.536463 30110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:01.542312 30110 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:37887
I20260806 22:43:01.542346 30237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:37887 every 8 connection(s)
I20260806 22:43:01.542706 30110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:43:01.545396 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:01.543409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:45859 (local address 127.19.74.254:34981)
0806 22:43:01.543470 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:01.543473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:01.543715 (+ 242us) server_negotiation.cc:407] Connection header received
0806 22:43:01.544125 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:01.544126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:01.544131 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:01.544183 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:01.545219 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:01.545220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:01.545221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:01.545224 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:01.545237 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:01.545242 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:01.545290 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:01.545332 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:43:01.545343 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:43:01.545614 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:01.543403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:34981 (local address 127.19.74.195:45859)
0806 22:43:01.543604 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:01.543634 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:01.544035 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:01.544230 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:01.544233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:01.544241 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:01.545062 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:01.545062 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:01.545087 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:01.545137 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:01.545285 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:01.545287 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:01.545290 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:01.545339 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:43:01.545383 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260806 22:43:01.546113 30238 heartbeater.cc:347] Connected to a master server at 127.19.74.254:34981
I20260806 22:43:01.546214 30238 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:01.546383 30238 heartbeater.cc:511] Master 127.19.74.254:34981 requested a full tablet report, sending...
I20260806 22:43:01.546777 29783 ts_manager.cc:194] Registered new tserver with Master: 02d36af6904541008898ffe584f6c3ad (127.19.74.195:37887)
I20260806 22:43:01.549256 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 30110
I20260806 22:43:01.549330 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:01.549537 19755 external_mini_cluster.cc:1428] Setting key 1a1bfc4ad4fe97b6817d63bf9f582e9c
I20260806 22:43:01.550407 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:43:01.552198 29834 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:43:01.551834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.19.74.254:34981)
0806 22:43:01.551870 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:01.551871 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:01.551971 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:43:01.552023 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:01.552024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:01.552028 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:01.552073 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:01.552155 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52562: BlockingRecv error: recv got EOF from 127.0.0.1:52562 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:43:01.552352 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 29843
I20260806 22:43:01.559391 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 29976
I20260806 22:43:01.565834 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 30110
I20260806 22:43:01.571187 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 29751
2026-08-06T22:43:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (545 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:43:01 dist-test-slave-pgkr krb5kdc[30248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:43:01 dist-test-slave-pgkr krb5kdc[30248](info): set up 2 sockets
Aug 06 22:43:01 dist-test-slave-pgkr krb5kdc[30248](info): commencing operation
krb5kdc: starting...
W20260806 22:43:02.629833 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:43:02 dist-test-slave-pgkr krb5kdc[30248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:43:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:43:02.762190 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:45891
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:45891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:02.849875 30261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:02.850102 30261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:02.850126 30261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:02.851305 30261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:02.851353 30261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:02.851370 30261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:02.851434 30261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:02.851464 30261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:02.853046 30261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:45891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:45891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:02.853327 30261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:02.853605 30261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:02.858695 30269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:02.858708 30266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:02.858708 30267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:02.858922 30261 server_base.cc:1034] running on GCE node
I20260806 22:43:02.859227 30261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:02.859521 30261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:02.860668 30261 hybrid_clock.cc:648] HybridClock initialized: now 1786056182860652 us; error 31 us; skew 500 ppm
Aug 06 22:43:02 dist-test-slave-pgkr krb5kdc[30248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056182, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:02.863200 30261 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:43:02.864048 30261 webserver.cc:460] Webserver started at http://127.19.74.254:43585/ using document root <none> and password file <none>
I20260806 22:43:02.864318 30261 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:02.864367 30261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:02.864576 30261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:02.865670 30261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "7d6babc7a0c449fabe3aad3656318f80"
format_stamp: "Formatted at 2026-08-06 22:43:02 on dist-test-slave-pgkr"
server_key: "0f58d6ef8f631b3d120be1da55d7c460"
server_key_iv: "a4e49656a33bb26fe25995fb2619e8c6"
server_key_version: "encryptionkey@0"
I20260806 22:43:02.866063 30261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "7d6babc7a0c449fabe3aad3656318f80"
format_stamp: "Formatted at 2026-08-06 22:43:02 on dist-test-slave-pgkr"
server_key: "0f58d6ef8f631b3d120be1da55d7c460"
server_key_iv: "a4e49656a33bb26fe25995fb2619e8c6"
server_key_version: "encryptionkey@0"
I20260806 22:43:02.867774 30261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:02.868568 30276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:02.868788 30261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:02.868911 30261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "7d6babc7a0c449fabe3aad3656318f80"
format_stamp: "Formatted at 2026-08-06 22:43:02 on dist-test-slave-pgkr"
server_key: "0f58d6ef8f631b3d120be1da55d7c460"
server_key_iv: "a4e49656a33bb26fe25995fb2619e8c6"
server_key_version: "encryptionkey@0"
I20260806 22:43:02.868971 30261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:02.903962 30261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:02.906416 30261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:02.906533 30261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:02.910748 30261 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:45891
I20260806 22:43:02.910830 30332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:45891 every 8 connection(s)
I20260806 22:43:02.911166 30261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:02.912225 30333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:02.914650 30333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80: Bootstrap starting.
I20260806 22:43:02.915289 30333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:02.915588 30333 log.cc:824] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:02.916307 30333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80: No bootstrap required, opened a new log
I20260806 22:43:02.917829 30333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6babc7a0c449fabe3aad3656318f80" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45891 } }
I20260806 22:43:02.917999 30333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:02.918044 30333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6babc7a0c449fabe3aad3656318f80, State: Initialized, Role: FOLLOWER
I20260806 22:43:02.918215 30333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6babc7a0c449fabe3aad3656318f80" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45891 } }
I20260806 22:43:02.918313 30333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:02.918356 30333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:02.918401 30333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:02.919332 30333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6babc7a0c449fabe3aad3656318f80" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45891 } }
I20260806 22:43:02.919456 30333 leader_election.cc:302] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d6babc7a0c449fabe3aad3656318f80; no voters:
I20260806 22:43:02.919473 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 30261
I20260806 22:43:02.919569 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:43:02.919658 30333 leader_election.cc:288] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:02.919703 30336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:02.919811 19755 external_mini_cluster.cc:1428] Setting key 2572fcc5a54931173821cbf07ffdee4a
I20260806 22:43:02.919836 30336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [term 1 LEADER]: Becoming Leader. State: Replica: 7d6babc7a0c449fabe3aad3656318f80, State: Running, Role: LEADER
I20260806 22:43:02.919916 30336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6babc7a0c449fabe3aad3656318f80" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45891 } }
I20260806 22:43:02.919961 30333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:02.920382 30337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d6babc7a0c449fabe3aad3656318f80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6babc7a0c449fabe3aad3656318f80" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45891 } } }
I20260806 22:43:02.920481 30337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:02.920363 30338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d6babc7a0c449fabe3aad3656318f80. Latest consensus state: current_term: 1 leader_uuid: "7d6babc7a0c449fabe3aad3656318f80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6babc7a0c449fabe3aad3656318f80" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45891 } } }
I20260806 22:43:02.920765 30338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:02.921061 30345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:02.921697 30345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:02.923316 30345 catalog_manager.cc:1347] Generated new cluster ID: 9cb26634860a40a79011f67817a9fc72
I20260806 22:43:02.923367 30345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:43:02 dist-test-slave-pgkr krb5kdc[30248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:43:02.932067 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:02.920360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.19.74.254:45891)
0806 22:43:02.920651 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:02.920662 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:02.920868 (+ 206us) server_negotiation.cc:407] Connection header received
0806 22:43:02.921117 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:02.921127 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:02.921241 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:02.921361 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:02.922382 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:02.923760 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:02.924790 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:02.925406 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:02.925529 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:02.928255 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:02.928259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:02.928260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:02.928273 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:02.930321 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:02.930700 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:02.930703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:02.930703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:02.930819 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:02.931030 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:02.931031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:02.931031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:02.931360 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:02.931521 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:02.931589 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:43:02.931837 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":76,"threads_started":1}
I20260806 22:43:02.938952 30345 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:02.939431 30345 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:02.943892 30345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80: Generated new TSK 0
I20260806 22:43:02.944072 30345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:43:02.976890 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45891
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:03.059170 30359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:03.059386 30359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:03.059427 30359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:03.060596 30359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:03.060662 30359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:03.060708 30359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:43:03.062198 30359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:03.062472 30359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:03.062758 30359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:03.067659 30364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:03.067643 30367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:03.067688 30365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:03.067689 30359 server_base.cc:1034] running on GCE node
I20260806 22:43:03.068049 30359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:03.068305 30359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:03.069448 30359 hybrid_clock.cc:648] HybridClock initialized: now 1786056183069441 us; error 29 us; skew 500 ppm
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056183, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:03.072073 30359 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:43:03.072474 30359 webserver.cc:460] Webserver started at http://127.19.74.193:43787/ using document root <none> and password file <none>
I20260806 22:43:03.072696 30359 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:03.072736 30359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:03.072829 30359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:03.073798 30359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "9574e1db1c9049f897663ead88b3f4aa"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "39fd2e7f4ea7486a63b3d6d80bd5ca3c"
server_key_iv: "ab8aabd67691b22e8a458fd65519fb7a"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.074100 30359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "9574e1db1c9049f897663ead88b3f4aa"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "39fd2e7f4ea7486a63b3d6d80bd5ca3c"
server_key_iv: "ab8aabd67691b22e8a458fd65519fb7a"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.075562 30359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:03.076756 30374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.077028 30359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:03.077127 30359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "9574e1db1c9049f897663ead88b3f4aa"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "39fd2e7f4ea7486a63b3d6d80bd5ca3c"
server_key_iv: "ab8aabd67691b22e8a458fd65519fb7a"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.077208 30359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:03.102425 30359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:03.104786 30359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:03.104986 30359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:03.105209 30359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:03.105509 30359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:03.105559 30359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.105624 30359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:03.105674 30359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.112138 30359 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:43023
I20260806 22:43:03.112202 30488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:43023 every 8 connection(s)
I20260806 22:43:03.112507 30359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056183, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:43:03.121779 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.113261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:59261 (local address 127.19.74.254:45891)
0806 22:43:03.113316 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.113318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.113542 (+ 224us) server_negotiation.cc:407] Connection header received
0806 22:43:03.114000 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.114002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.114013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.114085 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:03.114998 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.115521 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.116445 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.116663 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.116747 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.118775 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:03.118776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:03.118777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:03.118783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:03.120289 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:03.120795 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:03.120796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:03.120797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:03.120861 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:03.121098 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:03.121098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:03.121099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:03.121242 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:03.121296 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.121508 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.121700 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:43:03.121811 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.113208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45891 (local address 127.19.74.193:59261)
0806 22:43:03.113363 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.113416 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:03.113913 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:03.114139 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:03.114142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:03.114737 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:03.114924 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.114929 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.115599 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.115599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.116357 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.116359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.116772 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.116773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.116862 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.117322 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:03.117330 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:03.118685 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:03.120363 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:03.120369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:03.120371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:03.120694 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:03.120925 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:03.120926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:03.120926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:03.121027 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:03.121298 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:03.121302 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:03.121409 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 22:43:03.121484 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:43:03.121599 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260806 22:43:03.122488 30489 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45891
I20260806 22:43:03.122565 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 30359
I20260806 22:43:03.122602 30489 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:03.122689 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:43:03.122839 30489 heartbeater.cc:511] Master 127.19.74.254:45891 requested a full tablet report, sending...
I20260806 22:43:03.122927 19755 external_mini_cluster.cc:1428] Setting key 13d70455648d62404999fcf221ffe016
I20260806 22:43:03.123473 30294 ts_manager.cc:194] Registered new tserver with Master: 9574e1db1c9049f897663ead88b3f4aa (127.19.74.193:43023)
I20260806 22:43:03.124154 30294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:59261
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:43:03.147368 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45891
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:03.229153 30494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:03.229370 30494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:03.229413 30494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:03.230537 30494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:03.230598 30494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:03.230655 30494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:43:03.232093 30494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:03.232378 30494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:03.232611 30494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:03.237548 30502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:03.237648 30494 server_base.cc:1034] running on GCE node
W20260806 22:43:03.237516 30500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:03.237558 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:03.237929 30494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:03.238191 30494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:03.239328 30494 hybrid_clock.cc:648] HybridClock initialized: now 1786056183239317 us; error 27 us; skew 500 ppm
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056183, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:03.241760 30494 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:43:03.242204 30494 webserver.cc:460] Webserver started at http://127.19.74.194:40365/ using document root <none> and password file <none>
I20260806 22:43:03.242453 30494 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:03.242520 30494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:03.242693 30494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:03.243742 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "e6327028655048c7bf96fabfa437f3a3"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "4e0b9f60c62794ba780009d741fabdf2"
server_key_iv: "4d77c5cf1f81959807d83afd19e42c0e"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.244107 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "e6327028655048c7bf96fabfa437f3a3"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "4e0b9f60c62794ba780009d741fabdf2"
server_key_iv: "4d77c5cf1f81959807d83afd19e42c0e"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.246058 30494 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:43:03.247099 30509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.247275 30494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:43:03.247382 30494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "e6327028655048c7bf96fabfa437f3a3"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "4e0b9f60c62794ba780009d741fabdf2"
server_key_iv: "4d77c5cf1f81959807d83afd19e42c0e"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.247455 30494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:03.263895 30494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:03.265671 30494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:03.265827 30494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:03.266049 30494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:03.266366 30494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:03.266427 30494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.266470 30494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:03.266497 30494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.272332 30494 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:37707
I20260806 22:43:03.272385 30624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:37707 every 8 connection(s)
I20260806 22:43:03.272688 30494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056183, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:43:03.281308 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.273449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:41503 (local address 127.19.74.254:45891)
0806 22:43:03.273509 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.273511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.273710 (+ 199us) server_negotiation.cc:407] Connection header received
0806 22:43:03.274179 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.274180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.274190 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.274247 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:03.275184 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.275702 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.276636 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.276834 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.276891 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.278939 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:03.278941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:03.278941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:03.278948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:03.280159 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:03.280507 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:03.280508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:03.280508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:03.280558 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:03.280765 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:03.280766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:03.280766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:03.280864 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:03.280922 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.281170 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.281242 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:43:03.281450 30627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.273424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45891 (local address 127.19.74.194:41503)
0806 22:43:03.273581 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.273622 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:03.274099 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:03.274299 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:03.274302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:03.274923 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:03.275112 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.275117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.275783 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.275784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.276557 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.276559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.276910 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.276910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.277014 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.277441 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:03.277449 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:03.278839 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:03.280231 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:03.280237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:03.280238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:03.280443 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:03.280617 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:03.280617 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:03.280618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:03.280712 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:03.280937 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:03.280941 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:03.281049 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 22:43:03.281143 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:43:03.281258 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260806 22:43:03.281958 30625 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45891
I20260806 22:43:03.282068 30625 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:03.282258 30625 heartbeater.cc:511] Master 127.19.74.254:45891 requested a full tablet report, sending...
I20260806 22:43:03.282320 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 30494
I20260806 22:43:03.282467 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:03.282729 19755 external_mini_cluster.cc:1428] Setting key 6421b54aec0dbe90522a23fd6bd097d8
I20260806 22:43:03.282728 30294 ts_manager.cc:194] Registered new tserver with Master: e6327028655048c7bf96fabfa437f3a3 (127.19.74.194:37707)
I20260806 22:43:03.283128 30294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:41503
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:43:03.307363 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45891
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:03.387580 30630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:03.387773 30630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:03.387794 30630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:03.388831 30630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:03.388873 30630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:03.388898 30630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:43:03.390288 30630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:45891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:03.390460 30630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:03.390694 30630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:03.395494 30636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:03.395519 30635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:03.395525 30638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:03.395789 30630 server_base.cc:1034] running on GCE node
I20260806 22:43:03.396045 30630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:03.396282 30630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:03.397450 30630 hybrid_clock.cc:648] HybridClock initialized: now 1786056183397435 us; error 42 us; skew 500 ppm
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056183, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:03.399888 30630 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:43:03.400287 30630 webserver.cc:460] Webserver started at http://127.19.74.195:36431/ using document root <none> and password file <none>
I20260806 22:43:03.400461 30630 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:03.400498 30630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:03.400584 30630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:03.401520 30630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "4bf04e74d887431594a585e6feabc429"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "0b07761a12f1e61826cc046327b05b08"
server_key_iv: "a93d822669e09f875a19e3052c07e6e2"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.401865 30630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "4bf04e74d887431594a585e6feabc429"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "0b07761a12f1e61826cc046327b05b08"
server_key_iv: "a93d822669e09f875a19e3052c07e6e2"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.403216 30630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:03.403899 30645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.404104 30630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:03.404199 30630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "4bf04e74d887431594a585e6feabc429"
format_stamp: "Formatted at 2026-08-06 22:43:03 on dist-test-slave-pgkr"
server_key: "0b07761a12f1e61826cc046327b05b08"
server_key_iv: "a93d822669e09f875a19e3052c07e6e2"
server_key_version: "encryptionkey@0"
I20260806 22:43:03.404253 30630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:03.416669 30630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:03.418905 30630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:03.419031 30630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:03.419207 30630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:03.419449 30630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:03.419478 30630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.419499 30630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:03.419512 30630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:03.425390 30630 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:38029
I20260806 22:43:03.425487 30761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:38029 every 8 connection(s)
I20260806 22:43:03.425801 30630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056183, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:43:03.432194 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 30630
I20260806 22:43:03.432329 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:03.432535 19755 external_mini_cluster.cc:1428] Setting key 212d5c3038dbcc320ce62e490d9a7122
I20260806 22:43:03.435403 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.426530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:40973 (local address 127.19.74.254:45891)
0806 22:43:03.426597 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.426599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.426830 (+ 231us) server_negotiation.cc:407] Connection header received
0806 22:43:03.427369 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.427370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.427384 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.427433 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:03.428344 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.428840 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.429827 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.430046 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.430131 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.432452 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:03.432453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:03.432454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:03.432461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:03.433974 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:03.434395 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:03.434397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:03.434397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:03.434459 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:03.434764 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:03.434765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:03.434766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:03.434883 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:03.434959 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.435234 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.435317 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:43:03.435562 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.426512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:45891 (local address 127.19.74.195:40973)
0806 22:43:03.426715 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.426750 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:03.427254 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:03.427506 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:03.427510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:03.428057 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:03.428247 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.428251 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.428953 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.428954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.429735 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.429737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.430143 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.430144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.430272 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.430765 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:03.430774 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:03.432306 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:03.434076 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:03.434083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:03.434085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:03.434305 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:03.434524 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:03.434525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:03.434525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:03.434689 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:03.434950 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:03.434957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:03.435092 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:43:03.435191 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:43:03.435342 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260806 22:43:03.436017 30762 heartbeater.cc:347] Connected to a master server at 127.19.74.254:45891
I20260806 22:43:03.436115 30762 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:03.436286 30762 heartbeater.cc:511] Master 127.19.74.254:45891 requested a full tablet report, sending...
I20260806 22:43:03.436676 30293 ts_manager.cc:194] Registered new tserver with Master: 4bf04e74d887431594a585e6feabc429 (127.19.74.195:38029)
I20260806 22:43:03.437026 30293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:40973
I20260806 22:43:03.444116 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:43:03.451113 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.445320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.19.74.254:45891)
0806 22:43:03.445375 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.445377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.445488 (+ 111us) server_negotiation.cc:407] Connection header received
0806 22:43:03.445560 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.445561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.445571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.445640 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:03.446207 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.446770 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.447485 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.447673 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.447754 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.448825 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:03.448826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:03.448826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:03.448834 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:03.449903 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:03.450240 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:03.450241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:03.450242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:03.450300 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:03.450498 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:03.450499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:03.450499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:03.450711 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:03.450819 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.450923 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.450996 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:43:03.452544 30293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:43:03.454046 30293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:43:03.460206 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.456387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:37707 (local address 127.0.0.1:40882)
0806 22:43:03.456662 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.456669 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:03.456756 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:03.456938 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:03.456941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:03.456943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:03.457065 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.457068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.457867 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.457868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.459072 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.459075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.459883 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.459884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.459959 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.459972 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:43:03.460078 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:43:03.460131 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":72,"threads_started":1}
I20260806 22:43:03.460216 30777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.456488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40882 (local address 127.19.74.194:37707)
0806 22:43:03.456655 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.456661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.456763 (+ 102us) server_negotiation.cc:407] Connection header received
0806 22:43:03.456819 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.456821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.456833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.456914 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:03.457158 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.457775 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.459171 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.459783 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.459880 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.459940 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.460097 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.460136 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260806 22:43:03.460513 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.456191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:43023 (local address 127.0.0.1:37118)
0806 22:43:03.456328 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.456446 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:03.456512 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:03.456954 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:03.456955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:03.456957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:03.457025 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.457027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.458273 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.458273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.459418 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.459421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.460288 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.460289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.460365 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.460370 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:43:03.460399 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 22:43:03.460436 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260806 22:43:03.460848 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.456281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53156 (local address 127.19.74.195:38029)
0806 22:43:03.456868 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.456872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.456883 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:43:03.456912 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.456914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.456928 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.456991 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:03.457173 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.458163 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.459359 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.460040 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.460130 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.460185 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.460550 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.460776 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":98,"threads_started":1}
I20260806 22:43:03.460915 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.456592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.19.74.193:43023)
0806 22:43:03.456785 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.456789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.456800 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:43:03.456840 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.456842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.456857 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.457121 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:03.457194 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.458163 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.459501 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.460197 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.460689 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.460759 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.460784 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.460826 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260806 22:43:03.461011 30558 tablet_service.cc:1467] Processing CreateTablet for tablet c72323b1b2dc48beb4ed5486df287baf (DEFAULT_TABLE table=test-table [id=757e89c1fc0b4ad989121f37ba8e2d49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:03.461381 30558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c72323b1b2dc48beb4ed5486df287baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:03.461534 30423 tablet_service.cc:1467] Processing CreateTablet for tablet c72323b1b2dc48beb4ed5486df287baf (DEFAULT_TABLE table=test-table [id=757e89c1fc0b4ad989121f37ba8e2d49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:03.461862 30423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c72323b1b2dc48beb4ed5486df287baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:03.462733 30774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.456176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:38029 (local address 127.0.0.1:53156)
0806 22:43:03.456328 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.456366 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:03.456426 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:03.457056 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:03.457057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:03.457059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:03.457105 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.457106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.458297 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.458298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.459279 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.459282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.460225 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.460226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.460421 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.460424 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:03.460470 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:43:03.460500 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":15,"thread_start_us":63,"threads_started":1}
I20260806 22:43:03.463521 30694 tablet_service.cc:1467] Processing CreateTablet for tablet c72323b1b2dc48beb4ed5486df287baf (DEFAULT_TABLE table=test-table [id=757e89c1fc0b4ad989121f37ba8e2d49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:03.463824 30694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c72323b1b2dc48beb4ed5486df287baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:03.464367 30779 tablet_bootstrap.cc:492] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Bootstrap starting.
I20260806 22:43:03.464936 30781 tablet_bootstrap.cc:492] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa: Bootstrap starting.
I20260806 22:43:03.465144 30779 tablet_bootstrap.cc:654] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:03.465664 30781 tablet_bootstrap.cc:654] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:03.465953 30779 log.cc:824] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:03.466260 30781 log.cc:824] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:03.466878 30782 tablet_bootstrap.cc:492] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: Bootstrap starting.
I20260806 22:43:03.467130 30781 tablet_bootstrap.cc:492] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa: No bootstrap required, opened a new log
I20260806 22:43:03.467198 30781 ts_tablet_manager.cc:1397] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:03.467396 30782 tablet_bootstrap.cc:654] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:03.467860 30782 log.cc:824] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:03.468550 30779 tablet_bootstrap.cc:492] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: No bootstrap required, opened a new log
I20260806 22:43:03.468622 30782 tablet_bootstrap.cc:492] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: No bootstrap required, opened a new log
I20260806 22:43:03.468652 30779 ts_tablet_manager.cc:1397] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:43:03.468699 30782 ts_tablet_manager.cc:1397] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:43:03.469158 30781 raft_consensus.cc:348] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.469352 30781 raft_consensus.cc:374] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:03.469393 30781 raft_consensus.cc:729] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9574e1db1c9049f897663ead88b3f4aa, State: Initialized, Role: FOLLOWER
I20260806 22:43:03.469532 30781 consensus_queue.cc:260] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.469813 30781 ts_tablet_manager.cc:1428] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:43:03.469926 30489 heartbeater.cc:503] Master 127.19.74.254:45891 was elected leader, sending a full tablet report...
I20260806 22:43:03.470472 30779 raft_consensus.cc:348] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.470619 30779 raft_consensus.cc:374] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:03.470700 30779 raft_consensus.cc:729] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6327028655048c7bf96fabfa437f3a3, State: Initialized, Role: FOLLOWER
I20260806 22:43:03.470705 30782 raft_consensus.cc:348] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.470857 30782 raft_consensus.cc:374] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:03.470893 30782 raft_consensus.cc:729] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bf04e74d887431594a585e6feabc429, State: Initialized, Role: FOLLOWER
I20260806 22:43:03.470847 30779 consensus_queue.cc:260] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.471015 30782 consensus_queue.cc:260] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.471143 30779 ts_tablet_manager.cc:1428] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:43:03.471279 30782 ts_tablet_manager.cc:1428] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:43:03.471273 30625 heartbeater.cc:503] Master 127.19.74.254:45891 was elected leader, sending a full tablet report...
I20260806 22:43:03.471393 30762 heartbeater.cc:503] Master 127.19.74.254:45891 was elected leader, sending a full tablet report...
W20260806 22:43:03.523344 30626 tablet.cc:2369] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:43:03.540138 30786 raft_consensus.cc:482] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:03.540272 30786 raft_consensus.cc:504] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.540735 30786 leader_election.cc:288] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bf04e74d887431594a585e6feabc429 (127.19.74.195:38029), e6327028655048c7bf96fabfa437f3a3 (127.19.74.194:37707)
I20260806 22:43:03.543704 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.541071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:33521 (local address 127.19.74.195:38029)
0806 22:43:03.541127 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.541129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.541276 (+ 147us) server_negotiation.cc:407] Connection header received
0806 22:43:03.541342 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.541344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.541354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.541430 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:03.541583 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.542024 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.542965 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.543339 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.543425 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.543435 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.543577 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.543619 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:43:03.543705 30777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.541021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:35829 (local address 127.19.74.194:37707)
0806 22:43:03.541070 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.541072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.541084 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:43:03.541175 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.541177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.541195 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.541266 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:03.541433 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.541898 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.542967 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.543338 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.543426 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.543435 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.543581 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.543628 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:43:03.543732 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.540953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:37707 (local address 127.19.74.193:35829)
0806 22:43:03.541014 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.541026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:03.541073 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:03.541274 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:03.541275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:03.541276 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:03.541352 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.541353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.541981 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.541982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.542877 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.542880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.543423 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.543424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.543504 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.543507 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:03.543572 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:43:03.543619 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":60}
I20260806 22:43:03.543741 30789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.540961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:38029 (local address 127.19.74.193:33521)
0806 22:43:03.541196 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.541205 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:03.541283 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:03.541432 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:03.541433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:03.541435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:03.541513 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.541514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.542110 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.542111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.542876 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:03.542878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.543425 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.543426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:03.543503 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.543506 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:03.543572 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:43:03.543620 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":17,"thread_start_us":144,"threads_started":1}
I20260806 22:43:03.544179 30578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72323b1b2dc48beb4ed5486df287baf" candidate_uuid: "9574e1db1c9049f897663ead88b3f4aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6327028655048c7bf96fabfa437f3a3" is_pre_election: true
I20260806 22:43:03.544365 30578 raft_consensus.cc:2454] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9574e1db1c9049f897663ead88b3f4aa in term 0.
I20260806 22:43:03.544428 30716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72323b1b2dc48beb4ed5486df287baf" candidate_uuid: "9574e1db1c9049f897663ead88b3f4aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bf04e74d887431594a585e6feabc429" is_pre_election: true
I20260806 22:43:03.544611 30716 raft_consensus.cc:2454] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9574e1db1c9049f897663ead88b3f4aa in term 0.
I20260806 22:43:03.544636 30375 leader_election.cc:302] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9574e1db1c9049f897663ead88b3f4aa, e6327028655048c7bf96fabfa437f3a3; no voters:
I20260806 22:43:03.544782 30786 raft_consensus.cc:2781] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:43:03.544859 30786 raft_consensus.cc:482] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:43:03.544894 30786 raft_consensus.cc:3037] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:03.545720 30786 raft_consensus.cc:504] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.545873 30786 leader_election.cc:288] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [CANDIDATE]: Term 1 election: Requested vote from peers 4bf04e74d887431594a585e6feabc429 (127.19.74.195:38029), e6327028655048c7bf96fabfa437f3a3 (127.19.74.194:37707)
I20260806 22:43:03.546020 30578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72323b1b2dc48beb4ed5486df287baf" candidate_uuid: "9574e1db1c9049f897663ead88b3f4aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6327028655048c7bf96fabfa437f3a3"
I20260806 22:43:03.546070 30716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72323b1b2dc48beb4ed5486df287baf" candidate_uuid: "9574e1db1c9049f897663ead88b3f4aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bf04e74d887431594a585e6feabc429"
I20260806 22:43:03.546104 30578 raft_consensus.cc:3037] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:03.546139 30716 raft_consensus.cc:3037] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:03.547061 30716 raft_consensus.cc:2454] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9574e1db1c9049f897663ead88b3f4aa in term 1.
I20260806 22:43:03.547061 30578 raft_consensus.cc:2454] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9574e1db1c9049f897663ead88b3f4aa in term 1.
I20260806 22:43:03.547221 30375 leader_election.cc:302] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9574e1db1c9049f897663ead88b3f4aa, e6327028655048c7bf96fabfa437f3a3; no voters:
I20260806 22:43:03.547345 30786 raft_consensus.cc:2781] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:03.547544 30786 raft_consensus.cc:686] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [term 1 LEADER]: Becoming Leader. State: Replica: 9574e1db1c9049f897663ead88b3f4aa, State: Running, Role: LEADER
I20260806 22:43:03.547662 30786 consensus_queue.cc:237] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } }
I20260806 22:43:03.548527 30293 catalog_manager.cc:5495] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 9574e1db1c9049f897663ead88b3f4aa (127.19.74.193). New cstate: current_term: 1 leader_uuid: "9574e1db1c9049f897663ead88b3f4aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9574e1db1c9049f897663ead88b3f4aa" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260806 22:43:03.553527 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.551451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37128 (local address 127.19.74.193:43023)
0806 22:43:03.551489 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:03.551490 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:03.551497 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:43:03.551523 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:03.551524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:03.551538 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:03.551599 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:03.551739 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.552142 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.552847 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:03.553014 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:03.553061 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:03.553209 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:03.553293 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:43:03.553368 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:03.553404 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 22:43:03.553419 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:43:03.555147 30716 raft_consensus.cc:1261] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 1 FOLLOWER]: Refusing update from remote peer 9574e1db1c9049f897663ead88b3f4aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:03.555171 30578 raft_consensus.cc:1261] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 1 FOLLOWER]: Refusing update from remote peer 9574e1db1c9049f897663ead88b3f4aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:03.555467 30786 consensus_queue.cc:1035] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bf04e74d887431594a585e6feabc429" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 38029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:03.555598 30790 consensus_queue.cc:1035] T c72323b1b2dc48beb4ed5486df287baf P 9574e1db1c9049f897663ead88b3f4aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6327028655048c7bf96fabfa437f3a3" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:03.558015 30794 mvcc.cc:205] Tried to move back new op lower bound from 7315686127839469568 to 7315686127811870720. Current Snapshot: MvccSnapshot[applied={T|T < 7315686127839469568}]
I20260806 22:43:03.558820 30795 mvcc.cc:205] Tried to move back new op lower bound from 7315686127839469568 to 7315686127811870720. Current Snapshot: MvccSnapshot[applied={T|T < 7315686127839469568}]
I20260806 22:43:03.559995 30294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:03.560096 30294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:03.560827 30294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d6babc7a0c449fabe3aad3656318f80: Sending DeleteTablet for 3 replicas of tablet c72323b1b2dc48beb4ed5486df287baf
I20260806 22:43:03.561177 30694 tablet_service.cc:1514] Processing DeleteTablet for tablet c72323b1b2dc48beb4ed5486df287baf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:03 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:53156
I20260806 22:43:03.561389 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 30359
I20260806 22:43:03.561427 30803 tablet_replica.cc:330] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: stopping tablet replica
I20260806 22:43:03.561518 30803 raft_consensus.cc:2229] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:03.561610 30803 raft_consensus.cc:2258] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:03.562039 30803 ts_tablet_manager.cc:1905] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:03.562515 30558 tablet_service.cc:1514] Processing DeleteTablet for tablet c72323b1b2dc48beb4ed5486df287baf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:03 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:40882
I20260806 22:43:03.563944 30803 ts_tablet_manager.cc:1918] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:03.564011 30803 log.cc:1196] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/c72323b1b2dc48beb4ed5486df287baf
I20260806 22:43:03.564113 30804 tablet_replica.cc:330] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: stopping tablet replica
I20260806 22:43:03.564237 30804 raft_consensus.cc:2229] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:03.564347 30804 raft_consensus.cc:2258] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:03.564324 30803 ts_tablet_manager.cc:1939] T c72323b1b2dc48beb4ed5486df287baf P 4bf04e74d887431594a585e6feabc429: Deleting consensus metadata
I20260806 22:43:03.564736 30804 ts_tablet_manager.cc:1905] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:03.564766 30281 catalog_manager.cc:4841] TS 4bf04e74d887431594a585e6feabc429 (127.19.74.195:38029): tablet c72323b1b2dc48beb4ed5486df287baf (table test-table [id=757e89c1fc0b4ad989121f37ba8e2d49]) successfully deleted
I20260806 22:43:03.566048 30804 ts_tablet_manager.cc:1918] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:03.566145 30804 log.cc:1196] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/c72323b1b2dc48beb4ed5486df287baf
I20260806 22:43:03.566388 30804 ts_tablet_manager.cc:1939] T c72323b1b2dc48beb4ed5486df287baf P e6327028655048c7bf96fabfa437f3a3: Deleting consensus metadata
I20260806 22:43:03.566817 30278 catalog_manager.cc:4841] TS e6327028655048c7bf96fabfa437f3a3 (127.19.74.194:37707): tablet c72323b1b2dc48beb4ed5486df287baf (table test-table [id=757e89c1fc0b4ad989121f37ba8e2d49]) successfully deleted
W20260806 22:43:03.569391 30278 connection.cc:535] client connection to 127.19.74.193:43023 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:43:03.569494 30278 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:43:03.570047 30774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:03.569919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:43023 (local address 127.0.0.1:37130)
0806 22:43:03.569974 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:03.570015 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:43023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:43:03.570165 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 30494
W20260806 22:43:03.571300 30278 catalog_manager.cc:4584] TS 9574e1db1c9049f897663ead88b3f4aa (127.19.74.193:43023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c72323b1b2dc48beb4ed5486df287baf: Network error: Client connection negotiation failed: client connection to 127.19.74.193:43023: connect: Connection refused (error 111)
I20260806 22:43:03.575819 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 30630
I20260806 22:43:03.581759 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 30261
2026-08-06T22:43:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2011 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30808](info): set up 2 sockets
Aug 06 22:43:03 dist-test-slave-pgkr krb5kdc[30808](info): commencing operation
krb5kdc: starting...
W20260806 22:43:04.625432 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:43:04 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.74.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.74.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/rangeradmin_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/rangeradmin_127.19.74.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.74.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.74.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/rangerlookup_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/rangerlookup_127.19.74.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.74.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.74.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/HTTP_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/HTTP_127.19.74.212@KRBTEST.COM.keytab.
I20260806 22:43:04.789142 19755 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLdXImp/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/postgres -l logfile start
2026-08-06 22:43:07.139 UTC [30838] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:43:07.139 UTC [30838] LOG: listening on IPv4 address "127.19.74.212", port 41997
2026-08-06 22:43:07.142 UTC [30838] LOG: listening on Unix socket "/tmp/.s.PGSQL.41997"
2026-08-06 22:43:07.163 UTC [30841] LOG: database system was shut down at 2026-08-06 22:43:05 UTC
2026-08-06 22:43:07.167 UTC [30838] LOG: database system is ready to accept connections
I20260806 22:43:08.137952 19755 mini_postgres.cc:96] Postgres bound to 41997
2026-08-06 22:43:08.146 UTC [30849] FATAL: database "slave" does not exist
127.19.74.212:41997 - accepting connections
I20260806 22:43:08.146849 19755 mini_ranger.cc:161] Starting Ranger...
I20260806 22:43:08.162220 19755 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:43:08.595340 19755 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:43:08.595408 19755 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-admin
2026-08-06 22:43:08,945 [I] DB FLAVOR :POSTGRES
2026-08-06 22:43:08,946 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:43:08,946 [I] Checking connection..
2026-08-06 22:43:08,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:43:09,197 [I] Checking connection passed.
2026-08-06 22:43:09,197 [I] --------- Verifying version history table ---------
2026-08-06 22:43:09,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:43:09,463 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:43:09,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:43:09,717 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:43:09,717 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:43:09,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:43:09.966 UTC [30914] WARNING: there is no transaction in progress
2026-08-06 22:43:09,973 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:43:09,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:43:10,227 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:43:10,227 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:43:10,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:43:10,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:43:10,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pgkr.c.gcp-upstream.internal','N') ;"
2026-08-06 22:43:10,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:43:11,196 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:43:11,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:43:11,433 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:43:11,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:43:11,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:43:11,943 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:43:11,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:43:12.855 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:12.870 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:12.886 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:12.900 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:12.915 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:12.931 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:12.983 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:12.990 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13.003 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13.013 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13.023 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13.036 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13.046 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13.052 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13.058 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13.064 UTC [31024] WARNING: there is no transaction in progress
2026-08-06 22:43:13,640 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:43:13,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:43:13,892 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:43:13,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:43:14,150 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:43:14,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:43:14,422 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:43:14,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:43:14,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:43:14,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pgkr.c.gcp-upstream.internal';"
2026-08-06 22:43:15,157 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:43:15,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:43:15,418 [I] DB_PATCHES have already been applied
I20260806 22:43:15.423693 19755 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLdXImp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:43:15.423808 19755 mini_ranger.cc:193] Using host: 127.19.74.212
I20260806 22:43:15.428256 19755 mini_ranger.cc:227] Ranger admin URL: http://127.19.74.212:43651
Aug 06, 2026 10:43:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:43:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:43:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:43:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:43:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:43:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:43:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:43:16.539767 19755 mini_ranger.cc:160] Time spent starting Ranger: real 8.393s user 0.000s sys 0.010s
Aug 06, 2026 10:43:36 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:43:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:43:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:43:36 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:43:37 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:43:38.235266 19755 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.74.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.74.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/rangerkms_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/rangerkms_127.19.74.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.74.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.74.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.74.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/HTTP_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.74.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/HTTP_127.19.74.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:43:38 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056218, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:43:38.319555 19755 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:43:38.330163 19755 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:43:39.127009 19755 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:43:39.127096 19755 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms
I20260806 22:43:39.127116 19755 mini_ranger_kms.cc:227] Using postgres at 127.19.74.212:41997
2026-08-06 22:43:39,405 [I] DB FLAVOR :POSTGRES
2026-08-06 22:43:39,405 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:43:39,406 [I] Checking connection
2026-08-06 22:43:39,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:43:39,654 [I] connection success
2026-08-06 22:43:39,654 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:43:39,654 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:43:39,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:43:39,888 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:43:39,888 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:43:39,889 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:43:39,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:41997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:43:40,150 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:43:40.611502 19755 mini_ranger_kms.cc:310] Created kms service
I20260806 22:43:40.814437 19755 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:43:40.858283 19755 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:43:40 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056218, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:43:41.263983 19755 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:43:41.264110 19755 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:43:41.264134 19755 mini_ranger_kms.cc:240] Using host: 127.19.74.212
I20260806 22:43:41.269338 19755 mini_ranger_kms.cc:276] Ranger KMS PID: 31205
I20260806 22:43:41.269492 19755 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.74.212:46063
Aug 06, 2026 10:43:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:43:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:43:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:43:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:43:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:43:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 22:43:43.560379 19755 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.241s user 0.000s sys 0.261s
I20260806 22:43:43.560501 19755 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:43:43.560593 19755 mini_ranger_kms.cc:401] 127.19.74.212:46063/kms/v1/keys
Aug 06 22:43:43 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056223, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.74.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 22:43:44 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056223, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.74.212@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
Aug 06 22:43:44 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056223, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.74.212@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
Aug 06 22:43:46 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:43:46.749006 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37131
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:37131
--ranger_config_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46063/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:46.839689 31256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:46.839893 31256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:46.839915 31256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:46.841082 31256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:46.841122 31256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:46.841135 31256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:46.841146 31256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:46.841157 31256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:46.843178 31256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.74.212:46063/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:37131
--ranger_config_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:46.843444 31256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:46.843693 31256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:46.849469 31262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:46.849511 31264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:46.849575 31261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:46.849615 31256 server_base.cc:1034] running on GCE node
I20260806 22:43:46.849853 31256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:46.850171 31256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:46.851347 31256 hybrid_clock.cc:648] HybridClock initialized: now 1786056226851347 us; error 42 us; skew 500 ppm
Aug 06 22:43:46 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056226, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:46.854072 31256 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:43:46.854465 31256 webserver.cc:460] Webserver started at http://127.19.74.254:46279/ using document root <none> and password file <none>
I20260806 22:43:46.854773 31256 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:46.854817 31256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:46.854913 31256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:43:46 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056226, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:43:46.921185 31256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "bb1998349fa848b69ba864a6b057c142"
format_stamp: "Formatted at 2026-08-06 22:43:46 on dist-test-slave-pgkr"
server_key: "2d1bb69968ef8cfdaf84cd50ab2b63af"
server_key_iv: "383808272f9a4f35804ca6f8d1f18a2e"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:46.921813 31256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "bb1998349fa848b69ba864a6b057c142"
format_stamp: "Formatted at 2026-08-06 22:43:46 on dist-test-slave-pgkr"
server_key: "2d1bb69968ef8cfdaf84cd50ab2b63af"
server_key_iv: "383808272f9a4f35804ca6f8d1f18a2e"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:46.924211 31256 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:46.936117 31271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:46.936338 31256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:46.936442 31256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "bb1998349fa848b69ba864a6b057c142"
format_stamp: "Formatted at 2026-08-06 22:43:46 on dist-test-slave-pgkr"
server_key: "2d1bb69968ef8cfdaf84cd50ab2b63af"
server_key_iv: "383808272f9a4f35804ca6f8d1f18a2e"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:46.936555 31256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:46.945241 31256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:46.947489 31256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:46.947715 31256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:46.957372 31256 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:37131
I20260806 22:43:46.957629 31324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:37131 every 8 connection(s)
I20260806 22:43:46.957942 31256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:46.959024 31325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:46.962345 31325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142: Bootstrap starting.
I20260806 22:43:46.963478 31325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:46.964170 31325 log.cc:824] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:46.965167 31325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142: No bootstrap required, opened a new log
I20260806 22:43:46.966274 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 31256
I20260806 22:43:46.966382 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:43:46.966572 19755 external_mini_cluster.cc:1428] Setting key 07319cb342c5a6d785aee77a81014985
I20260806 22:43:46.969784 31325 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1998349fa848b69ba864a6b057c142" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37131 } }
I20260806 22:43:46.970054 31325 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:46.970144 31325 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1998349fa848b69ba864a6b057c142, State: Initialized, Role: FOLLOWER
I20260806 22:43:46.970327 31325 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1998349fa848b69ba864a6b057c142" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37131 } }
I20260806 22:43:46.970427 31325 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:46.970481 31325 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:46.970561 31325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:46.971312 31325 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1998349fa848b69ba864a6b057c142" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37131 } }
I20260806 22:43:46.971427 31325 leader_election.cc:302] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1998349fa848b69ba864a6b057c142; no voters:
I20260806 22:43:46.971611 31325 leader_election.cc:288] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:46.971709 31330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:46.971946 31330 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [term 1 LEADER]: Becoming Leader. State: Replica: bb1998349fa848b69ba864a6b057c142, State: Running, Role: LEADER
I20260806 22:43:46.972030 31325 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:46.972090 31330 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1998349fa848b69ba864a6b057c142" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37131 } }
I20260806 22:43:46.972622 31330 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb1998349fa848b69ba864a6b057c142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1998349fa848b69ba864a6b057c142" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37131 } } }
I20260806 22:43:46.972815 31330 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:46.973168 31330 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb1998349fa848b69ba864a6b057c142. Latest consensus state: current_term: 1 leader_uuid: "bb1998349fa848b69ba864a6b057c142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1998349fa848b69ba864a6b057c142" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37131 } } }
I20260806 22:43:46.973321 31330 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:46.974409 31337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:46.974956 31337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:46.975394 31325 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 22:43:46 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:43:46.978096 31337 catalog_manager.cc:1347] Generated new cluster ID: ad8f84125bea471ba9decccd4256a8a5
I20260806 22:43:46.978150 31337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:43:46.989585 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:46.967252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39508 (local address 127.19.74.254:37131)
0806 22:43:46.967537 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:46.967546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:46.967618 (+ 72us) server_negotiation.cc:407] Connection header received
0806 22:43:46.969729 (+ 2111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:46.969740 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:46.969803 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:46.969950 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:46.970742 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:46.972028 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:46.973168 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:46.973807 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:46.974219 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:46.977447 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:46.977451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:46.977451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:46.977477 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:46.986066 (+ 8589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:46.986735 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:46.986739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:46.986740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:46.986952 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:46.987199 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:46.987199 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:46.987200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:46.987945 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:46.988802 (+ 857us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:46.988877 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:43:46.989112 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":121,"threads_started":1}
I20260806 22:43:47.023891 31337 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:47.024753 31337 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:47.036944 31337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142: Generated new TSK 0
I20260806 22:43:47.037281 31337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:43:47.717648 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37131
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:47.836419 31371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:47.836628 31371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:47.836653 31371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:47.837857 31371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:47.837896 31371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:47.837922 31371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:43:47.839473 31371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.74.212:46063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:47.839712 31371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:47.839975 31371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:47.845985 31377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:47.846061 31379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:47.846171 31376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:47.846339 31371 server_base.cc:1034] running on GCE node
I20260806 22:43:47.846550 31371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:47.846823 31371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:47.847985 31371 hybrid_clock.cc:648] HybridClock initialized: now 1786056227847974 us; error 31 us; skew 500 ppm
Aug 06 22:43:47 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:47.851661 31371 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:43:47.852356 31371 webserver.cc:460] Webserver started at http://127.19.74.193:44249/ using document root <none> and password file <none>
I20260806 22:43:47.852666 31371 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:47.852741 31371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:47.852912 31371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:43:47 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:43:47.867033 31371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "b234a604e19546caad6e5d9b5e4c38e4"
format_stamp: "Formatted at 2026-08-06 22:43:47 on dist-test-slave-pgkr"
server_key: "8c06d5b70a56210b6493317ac0d0cc56"
server_key_iv: "b56d7c99850daa0d49cc46082c984874"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:47.867702 31371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "b234a604e19546caad6e5d9b5e4c38e4"
format_stamp: "Formatted at 2026-08-06 22:43:47 on dist-test-slave-pgkr"
server_key: "8c06d5b70a56210b6493317ac0d0cc56"
server_key_iv: "b56d7c99850daa0d49cc46082c984874"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:47.879518 31371 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.001s sys 0.000s
I20260806 22:43:47.897361 31386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:47.897723 31371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:43:47.897941 31371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "b234a604e19546caad6e5d9b5e4c38e4"
format_stamp: "Formatted at 2026-08-06 22:43:47 on dist-test-slave-pgkr"
server_key: "8c06d5b70a56210b6493317ac0d0cc56"
server_key_iv: "b56d7c99850daa0d49cc46082c984874"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:47.898204 31371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:47.908108 31371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:47.910327 31371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:47.910477 31371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:47.910784 31371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:47.911113 31371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:47.911171 31371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:47.911213 31371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:47.911242 31371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:47.926184 31371 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:45137
I20260806 22:43:47.926260 31499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:45137 every 8 connection(s)
I20260806 22:43:47.926683 31371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:47.935467 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 31371
I20260806 22:43:47.935599 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:43:47.935933 19755 external_mini_cluster.cc:1428] Setting key a62cff9d207c0b214eb91b50eafae67c
Aug 06 22:43:47 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:43:47.942046 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:47.927810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:53813 (local address 127.19.74.254:37131)
0806 22:43:47.930757 (+ 2947us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:47.930760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:47.930769 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:43:47.930794 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:47.930795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:47.930805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:47.930914 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:47.931712 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:47.932287 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:47.933471 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:47.933815 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:47.933889 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:47.937557 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:47.937558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:47.937559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:47.937567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:47.939680 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:47.940214 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:47.940215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:47.940216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:47.940281 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:47.940629 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:47.940630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:47.940630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:47.940858 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:47.940970 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:47.941750 (+ 780us) server_negotiation.cc:299] Negotiation successful
0806 22:43:47.941891 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2896,"thread_start_us":130,"threads_started":1}
I20260806 22:43:47.942705 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:47.927767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37131 (local address 127.19.74.193:53813)
0806 22:43:47.928067 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:47.928110 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:47.928867 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:47.931004 (+ 2137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:47.931010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:47.931334 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:47.931609 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:47.931614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:47.932387 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:47.932388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:47.933363 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:47.933366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:47.933965 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:47.933966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:47.934200 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:47.935068 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:47.935084 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:47.937354 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:47.939784 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:47.939793 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:47.939796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:47.940123 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:47.940352 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:47.940353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:47.940353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:47.940493 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:47.941251 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:47.941259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:47.941522 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 22:43:47.942192 (+ 670us) client_negotiation.cc:240] Negotiation successful
0806 22:43:47.942369 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":150,"threads_started":1}
I20260806 22:43:47.943259 31500 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37131
I20260806 22:43:47.943372 31500 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:47.943598 31500 heartbeater.cc:511] Master 127.19.74.254:37131 requested a full tablet report, sending...
I20260806 22:43:47.944197 31288 ts_manager.cc:194] Registered new tserver with Master: b234a604e19546caad6e5d9b5e4c38e4 (127.19.74.193:45137)
I20260806 22:43:47.944811 31288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:53813
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:43:47.971812 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37131
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:48.090603 31508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:48.090960 31508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:48.090998 31508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:48.092164 31508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:48.092253 31508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:48.092299 31508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:43:48.093812 31508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.74.212:46063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:48.094086 31508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:48.094313 31508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:48.099547 31518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:48.099547 31515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:48.099875 31516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:48.100687 31508 server_base.cc:1034] running on GCE node
I20260806 22:43:48.100935 31508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:48.101267 31508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:48.102447 31508 hybrid_clock.cc:648] HybridClock initialized: now 1786056228102437 us; error 27 us; skew 500 ppm
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:48.108312 31508 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:43:48.108850 31508 webserver.cc:460] Webserver started at http://127.19.74.194:39101/ using document root <none> and password file <none>
I20260806 22:43:48.109081 31508 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:48.109154 31508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:48.109298 31508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:43:48.130467 31508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4"
format_stamp: "Formatted at 2026-08-06 22:43:48 on dist-test-slave-pgkr"
server_key: "5df3fa76a427516e86845a0718e494ff"
server_key_iv: "a9ef77795255f095a6dda4020e6d928a"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:48.131068 31508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4"
format_stamp: "Formatted at 2026-08-06 22:43:48 on dist-test-slave-pgkr"
server_key: "5df3fa76a427516e86845a0718e494ff"
server_key_iv: "a9ef77795255f095a6dda4020e6d928a"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:48.132572 31508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:48.147454 31525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:48.147682 31508 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 22:43:48.147799 31508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4"
format_stamp: "Formatted at 2026-08-06 22:43:48 on dist-test-slave-pgkr"
server_key: "5df3fa76a427516e86845a0718e494ff"
server_key_iv: "a9ef77795255f095a6dda4020e6d928a"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:48.147920 31508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:48.164875 31508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:48.167169 31508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:48.167317 31508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:48.167655 31508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:48.168056 31508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:48.168107 31508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:48.168139 31508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:48.168164 31508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:48.180331 31508 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:38213
I20260806 22:43:48.180845 31508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:43:48.183429 31640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:38213 every 8 connection(s)
I20260806 22:43:48.188849 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 31508
I20260806 22:43:48.188968 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:48.189185 19755 external_mini_cluster.cc:1428] Setting key 77d9d05c8e0d7b44acae702d32cebed5
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:43:48.200459 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.185820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:57393 (local address 127.19.74.254:37131)
0806 22:43:48.185881 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.185884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.187242 (+ 1358us) server_negotiation.cc:407] Connection header received
0806 22:43:48.189085 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.189087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.189105 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.189149 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:48.190060 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.190828 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.192352 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.192687 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.192767 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.196694 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:48.196696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:48.196697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:48.196706 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:48.198512 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:48.199085 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:48.199087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:48.199087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:48.199149 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:48.199456 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:48.199458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:48.199458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:48.199617 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:48.199676 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.200226 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.200315 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:43:48.201318 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.185531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37131 (local address 127.19.74.194:57393)
0806 22:43:48.187096 (+ 1565us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.187149 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:48.188877 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:48.189390 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:48.189394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:48.189649 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:48.189941 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.189948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.190949 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.190950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.192227 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.192231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.192838 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.192839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.192988 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.194751 (+ 1763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:48.194763 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:48.196569 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:48.198646 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:48.198652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:48.198654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:48.198982 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:48.199226 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:48.199227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:48.199227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:48.199376 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:48.199854 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:48.199860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:48.200049 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 22:43:48.200892 (+ 843us) client_negotiation.cc:240] Negotiation successful
0806 22:43:48.201073 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
I20260806 22:43:48.202034 31641 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37131
I20260806 22:43:48.202138 31641 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:48.202312 31641 heartbeater.cc:511] Master 127.19.74.254:37131 requested a full tablet report, sending...
I20260806 22:43:48.202731 31288 ts_manager.cc:194] Registered new tserver with Master: c3d3bc553e4d4d73bcc834c8c75d1ed4 (127.19.74.194:38213)
I20260806 22:43:48.203231 31288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:57393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:43:48.221218 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37131
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:48.311003 31647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:48.311224 31647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:48.311260 31647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:48.312536 31647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:48.312588 31647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:48.312626 31647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:43:48.314323 31647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.74.212:46063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:43:48.314585 31647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:48.314837 31647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:48.320727 31656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:48.320894 31653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:48.320940 31654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:48.321262 31647 server_base.cc:1034] running on GCE node
I20260806 22:43:48.321537 31647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:48.321818 31647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:48.322964 31647 hybrid_clock.cc:648] HybridClock initialized: now 1786056228322957 us; error 31 us; skew 500 ppm
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:48.325537 31647 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:43:48.325990 31647 webserver.cc:460] Webserver started at http://127.19.74.195:36737/ using document root <none> and password file <none>
I20260806 22:43:48.326221 31647 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:48.326295 31647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:48.326428 31647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:43:48.345074 31647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "673747cbe6aa465b9c2ff63a3c2574eb"
format_stamp: "Formatted at 2026-08-06 22:43:48 on dist-test-slave-pgkr"
server_key: "3061c50efecda175a7a07fb6d4d27b30"
server_key_iv: "e4fa7288afaa6ed07daa54e70e7f2cda"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:48.345574 31647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "673747cbe6aa465b9c2ff63a3c2574eb"
format_stamp: "Formatted at 2026-08-06 22:43:48 on dist-test-slave-pgkr"
server_key: "3061c50efecda175a7a07fb6d4d27b30"
server_key_iv: "e4fa7288afaa6ed07daa54e70e7f2cda"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:48.347162 31647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:43:48.362751 31663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:48.363018 31647 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 22:43:48.363125 31647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "673747cbe6aa465b9c2ff63a3c2574eb"
format_stamp: "Formatted at 2026-08-06 22:43:48 on dist-test-slave-pgkr"
server_key: "3061c50efecda175a7a07fb6d4d27b30"
server_key_iv: "e4fa7288afaa6ed07daa54e70e7f2cda"
server_key_version: "kuduclusterkey@0"
I20260806 22:43:48.363267 31647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:48.377547 31647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:48.380151 31647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:48.380321 31647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:48.380553 31647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:48.380934 31647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:48.381003 31647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:48.381053 31647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:48.381083 31647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:48.389273 31647 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:43675
I20260806 22:43:48.389411 31777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:43675 every 8 connection(s)
I20260806 22:43:48.389707 31647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:43:48.398140 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 31647
I20260806 22:43:48.398252 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:48.398427 19755 external_mini_cluster.cc:1428] Setting key 1a4bef24d4e78b5f8d8a559cfef8511a
I20260806 22:43:48.401119 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.390531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:56623 (local address 127.19.74.254:37131)
0806 22:43:48.390581 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.390583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.390848 (+ 265us) server_negotiation.cc:407] Connection header received
0806 22:43:48.391586 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.391588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.391600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.391671 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:48.392403 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.393212 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.394759 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.395066 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.395148 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.397697 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:48.397698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:48.397699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:48.397706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:48.399508 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:48.400003 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:48.400004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:48.400005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:48.400072 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:48.400394 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:48.400395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:48.400395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:48.400533 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:48.400603 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.400943 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.401025 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:43:48.401306 31780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.390508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37131 (local address 127.19.74.195:56623)
0806 22:43:48.390716 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.390763 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:48.391436 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:48.391745 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:48.391749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:48.391993 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:48.392286 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.392293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.393323 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.393324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.394646 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.394649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.395179 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.395180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.395329 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.395932 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:48.395942 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:48.397591 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:48.399607 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:48.399614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:48.399616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:48.399911 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:48.400158 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:48.400159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:48.400160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:48.400316 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:48.400619 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:48.400625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:48.400779 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 22:43:48.400902 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 22:43:48.401074 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260806 22:43:48.401824 31778 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37131
I20260806 22:43:48.401921 31778 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:48.402096 31778 heartbeater.cc:511] Master 127.19.74.254:37131 requested a full tablet report, sending...
I20260806 22:43:48.402510 31288 ts_manager.cc:194] Registered new tserver with Master: 673747cbe6aa465b9c2ff63a3c2574eb (127.19.74.195:43675)
I20260806 22:43:48.403039 31288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:56623
I20260806 22:43:48.410300 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:43:48.419925 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.412083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39518 (local address 127.19.74.254:37131)
0806 22:43:48.412119 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.412121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.412381 (+ 260us) server_negotiation.cc:407] Connection header received
0806 22:43:48.412476 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.412477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.412487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.412529 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:48.413316 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.414039 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.415146 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.415437 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.415478 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.416981 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:48.416983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:48.416983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:48.416989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:48.418711 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:48.419096 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:48.419097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:48.419098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:48.419150 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:48.419341 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:48.419342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:48.419342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:48.419459 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:48.419544 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.419758 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.419833 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:43:48.422029 31288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:43:48.424335 31288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:43:48.433593 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.427369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:45137 (local address 127.0.0.1:35330)
0806 22:43:48.428352 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.428360 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:48.428395 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:48.428776 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:48.428779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:48.428781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:48.428846 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.428847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.430526 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.430527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.431792 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.431795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.433286 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.433287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.433370 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.433383 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:43:48.433453 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:43:48.433515 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"thread_start_us":98,"threads_started":1}
I20260806 22:43:48.439571 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.427626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43675 (local address 127.0.0.1:56876)
0806 22:43:48.433967 (+ 6341us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.433978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:48.434025 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:48.434301 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:48.434302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:48.434303 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:48.434343 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.434344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.436197 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.436198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.437882 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.437884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.439283 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.439284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.439348 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.439351 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:48.439402 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:43:48.439440 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6330,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260806 22:43:48.440466 31795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.427845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56876 (local address 127.19.74.195:43675)
0806 22:43:48.428710 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.428714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.434086 (+ 5372us) server_negotiation.cc:407] Connection header received
0806 22:43:48.434138 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.434141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.434156 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.434226 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:48.434410 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.436101 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.437981 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.439075 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.439155 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.439245 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.440323 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.440387 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":82,"threads_started":1}
I20260806 22:43:48.440858 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.427678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35330 (local address 127.19.74.193:45137)
0806 22:43:48.428494 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.428499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.428521 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:43:48.428564 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.428566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.428582 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.428925 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:48.429003 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.430429 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.432271 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.433079 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.433154 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.433248 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.440727 (+ 7479us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.440776 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":60,"threads_started":1}
I20260806 22:43:48.441812 31711 tablet_service.cc:1467] Processing CreateTablet for tablet 501195803f6848dca02dfed17a624055 (DEFAULT_TABLE table=test-table [id=3b1a947d8b1b40db8e819a8e8e79943a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:48.442068 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.427363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:38213 (local address 127.0.0.1:38314)
0806 22:43:48.428159 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.428198 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:48.428252 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:48.435102 (+ 6850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:48.435103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:48.435105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:48.435163 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.435164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.436548 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.436549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.437749 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.437752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.441794 (+ 4042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.441795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.441873 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.441878 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:43:48.441938 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:43:48.441977 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":25,"thread_start_us":112,"threads_started":1}
I20260806 22:43:48.442242 31711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501195803f6848dca02dfed17a624055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:48.442315 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.427469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.19.74.194:38213)
0806 22:43:48.434758 (+ 7289us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.434763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.434776 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:43:48.434815 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.434818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.434834 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.434935 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:48.435250 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.436431 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.439973 (+ 3542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.441678 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.441776 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.441853 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.441956 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.441993 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7247,"thread_start_us":71,"threads_started":1}
I20260806 22:43:48.443095 31433 tablet_service.cc:1467] Processing CreateTablet for tablet 501195803f6848dca02dfed17a624055 (DEFAULT_TABLE table=test-table [id=3b1a947d8b1b40db8e819a8e8e79943a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:48.443089 31555 tablet_service.cc:1467] Processing CreateTablet for tablet 501195803f6848dca02dfed17a624055 (DEFAULT_TABLE table=test-table [id=3b1a947d8b1b40db8e819a8e8e79943a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:48.443482 31433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501195803f6848dca02dfed17a624055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:48.443482 31555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501195803f6848dca02dfed17a624055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:48.445926 31796 tablet_bootstrap.cc:492] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: Bootstrap starting.
I20260806 22:43:48.447041 31796 tablet_bootstrap.cc:654] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:48.447340 31796 log.cc:824] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:48.447430 31798 tablet_bootstrap.cc:492] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4: Bootstrap starting.
I20260806 22:43:48.448046 31798 tablet_bootstrap.cc:654] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:48.448153 31796 tablet_bootstrap.cc:492] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: No bootstrap required, opened a new log
I20260806 22:43:48.448217 31796 ts_tablet_manager.cc:1397] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:48.448369 31798 log.cc:824] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:48.448696 31797 tablet_bootstrap.cc:492] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: Bootstrap starting.
I20260806 22:43:48.449091 31798 tablet_bootstrap.cc:492] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4: No bootstrap required, opened a new log
I20260806 22:43:48.449149 31798 ts_tablet_manager.cc:1397] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:43:48.449617 31797 tablet_bootstrap.cc:654] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:48.449904 31797 log.cc:824] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:48.450340 31796 raft_consensus.cc:348] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.450493 31796 raft_consensus.cc:374] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:48.450529 31796 raft_consensus.cc:729] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 673747cbe6aa465b9c2ff63a3c2574eb, State: Initialized, Role: FOLLOWER
I20260806 22:43:48.450577 31797 tablet_bootstrap.cc:492] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: No bootstrap required, opened a new log
I20260806 22:43:48.450650 31797 ts_tablet_manager.cc:1397] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:48.450667 31796 consensus_queue.cc:260] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.450925 31796 ts_tablet_manager.cc:1428] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:43:48.451308 31778 heartbeater.cc:503] Master 127.19.74.254:37131 was elected leader, sending a full tablet report...
I20260806 22:43:48.451336 31798 raft_consensus.cc:348] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.451484 31798 raft_consensus.cc:374] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:48.451514 31798 raft_consensus.cc:729] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b234a604e19546caad6e5d9b5e4c38e4, State: Initialized, Role: FOLLOWER
I20260806 22:43:48.451656 31798 consensus_queue.cc:260] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.451894 31798 ts_tablet_manager.cc:1428] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:43:48.452000 31500 heartbeater.cc:503] Master 127.19.74.254:37131 was elected leader, sending a full tablet report...
I20260806 22:43:48.452736 31797 raft_consensus.cc:348] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.452912 31797 raft_consensus.cc:374] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:48.452960 31797 raft_consensus.cc:729] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d3bc553e4d4d73bcc834c8c75d1ed4, State: Initialized, Role: FOLLOWER
I20260806 22:43:48.453076 31797 consensus_queue.cc:260] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.453346 31797 ts_tablet_manager.cc:1428] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:43:48.453398 31641 heartbeater.cc:503] Master 127.19.74.254:37131 was elected leader, sending a full tablet report...
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
W20260806 22:43:48.640584 31779 tablet.cc:2369] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:43:48.679451 31501 tablet.cc:2369] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:43:48.685335 31642 tablet.cc:2369] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:43:48.836061 31803 raft_consensus.cc:482] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:48.836205 31803 raft_consensus.cc:504] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.836606 31803 leader_election.cc:288] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 673747cbe6aa465b9c2ff63a3c2574eb (127.19.74.195:43675), c3d3bc553e4d4d73bcc834c8c75d1ed4 (127.19.74.194:38213)
I20260806 22:43:48.841972 31795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.837460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:32897 (local address 127.19.74.195:43675)
0806 22:43:48.837554 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.837556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.838194 (+ 638us) server_negotiation.cc:407] Connection header received
0806 22:43:48.838490 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.838492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.838504 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.838586 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:48.838984 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.839681 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.841146 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.841609 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.841657 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.841666 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.841873 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.841907 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:43:48.842257 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.836880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43675 (local address 127.19.74.193:32897)
0806 22:43:48.838113 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.838126 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:48.838378 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:48.838765 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:48.838766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:48.838769 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:48.838876 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.838878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.839815 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.839816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.841003 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.841007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.841739 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.841739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.841799 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.841802 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:48.842097 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 22:43:48.842135 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1198,"mutex_wait_us":18,"thread_start_us":90,"threads_started":1}
I20260806 22:43:48.842947 31732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501195803f6848dca02dfed17a624055" candidate_uuid: "b234a604e19546caad6e5d9b5e4c38e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" is_pre_election: true
I20260806 22:43:48.843128 31732 raft_consensus.cc:2454] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b234a604e19546caad6e5d9b5e4c38e4 in term 0.
I20260806 22:43:48.843420 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.837508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:48445 (local address 127.19.74.194:38213)
0806 22:43:48.837589 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.837591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.838569 (+ 978us) server_negotiation.cc:407] Connection header received
0806 22:43:48.838683 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.838684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.838694 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.838723 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:48.839187 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.840107 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.841872 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.842429 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.842476 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.842484 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.843311 (+ 827us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.843352 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:43:48.843406 31387 leader_election.cc:302] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 673747cbe6aa465b9c2ff63a3c2574eb, b234a604e19546caad6e5d9b5e4c38e4; no voters:
I20260806 22:43:48.843549 31803 raft_consensus.cc:2781] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:43:48.843604 31803 raft_consensus.cc:482] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:43:48.843633 31803 raft_consensus.cc:3037] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:48.844621 31803 raft_consensus.cc:504] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.844770 31803 leader_election.cc:288] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [CANDIDATE]: Term 1 election: Requested vote from peers 673747cbe6aa465b9c2ff63a3c2574eb (127.19.74.195:43675), c3d3bc553e4d4d73bcc834c8c75d1ed4 (127.19.74.194:38213)
Aug 06 22:43:48 dist-test-slave-pgkr krb5kdc[30808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056228, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:43:48.845033 31806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.837104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:38213 (local address 127.19.74.193:48445)
0806 22:43:48.838499 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.838509 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:48.838594 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:48.839038 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:48.839039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:48.839041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:48.839110 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.839112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.840357 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.840358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.841475 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:48.841478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.843106 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.843106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:48.843201 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.843204 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:48.844802 (+ 1598us) client_negotiation.cc:240] Negotiation successful
0806 22:43:48.844890 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1378,"thread_start_us":72,"threads_started":1}
I20260806 22:43:48.844985 31732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501195803f6848dca02dfed17a624055" candidate_uuid: "b234a604e19546caad6e5d9b5e4c38e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "673747cbe6aa465b9c2ff63a3c2574eb"
I20260806 22:43:48.845077 31732 raft_consensus.cc:3037] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:48.845947 31732 raft_consensus.cc:2454] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b234a604e19546caad6e5d9b5e4c38e4 in term 1.
I20260806 22:43:48.846211 31387 leader_election.cc:302] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 673747cbe6aa465b9c2ff63a3c2574eb, b234a604e19546caad6e5d9b5e4c38e4; no voters:
I20260806 22:43:48.846289 31803 raft_consensus.cc:2781] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:48.846415 31803 raft_consensus.cc:686] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 1 LEADER]: Becoming Leader. State: Replica: b234a604e19546caad6e5d9b5e4c38e4, State: Running, Role: LEADER
I20260806 22:43:48.846374 31582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501195803f6848dca02dfed17a624055" candidate_uuid: "b234a604e19546caad6e5d9b5e4c38e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" is_pre_election: true
I20260806 22:43:48.846508 31582 raft_consensus.cc:2454] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b234a604e19546caad6e5d9b5e4c38e4 in term 0.
I20260806 22:43:48.846529 31803 consensus_queue.cc:237] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } }
I20260806 22:43:48.846632 31583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501195803f6848dca02dfed17a624055" candidate_uuid: "b234a604e19546caad6e5d9b5e4c38e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4"
I20260806 22:43:48.846684 31583 raft_consensus.cc:3037] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:48.847366 31287 catalog_manager.cc:5495] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b234a604e19546caad6e5d9b5e4c38e4 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "b234a604e19546caad6e5d9b5e4c38e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b234a604e19546caad6e5d9b5e4c38e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 } health_report { overall_health: UNKNOWN } } }
I20260806 22:43:48.847488 31583 raft_consensus.cc:2454] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b234a604e19546caad6e5d9b5e4c38e4 in term 1.
I20260806 22:43:48.875557 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.871915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.19.74.193:45137)
0806 22:43:48.872103 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:48.872105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:48.872112 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:43:48.872148 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:48.872149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:48.872171 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:48.872257 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:48.872444 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.873178 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.874317 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:48.874655 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:48.874729 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:48.874922 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:48.875059 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:43:48.875139 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:48.875369 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:43:48.875389 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260806 22:43:48.877779 31583 raft_consensus.cc:1261] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 1 FOLLOWER]: Refusing update from remote peer b234a604e19546caad6e5d9b5e4c38e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:48.877794 31732 raft_consensus.cc:1261] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 1 FOLLOWER]: Refusing update from remote peer b234a604e19546caad6e5d9b5e4c38e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:48.878139 31808 consensus_queue.cc:1035] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "673747cbe6aa465b9c2ff63a3c2574eb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:48.878245 31803 consensus_queue.cc:1035] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d3bc553e4d4d73bcc834c8c75d1ed4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:48.888955 31813 mvcc.cc:205] Tried to move back new op lower bound from 7315686313479925760 to 7315686313355890688. Current Snapshot: MvccSnapshot[applied={T|T < 7315686313479925760}]
I20260806 22:43:48.889107 31811 mvcc.cc:205] Tried to move back new op lower bound from 7315686313479925760 to 7315686313355890688. Current Snapshot: MvccSnapshot[applied={T|T < 7315686313479925760}]
I20260806 22:43:48.892760 31288 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:48.892853 31288 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:48.893654 31288 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb1998349fa848b69ba864a6b057c142: Sending DeleteTablet for 3 replicas of tablet 501195803f6848dca02dfed17a624055
I20260806 22:43:48.894043 31555 tablet_service.cc:1514] Processing DeleteTablet for tablet 501195803f6848dca02dfed17a624055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:48 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:38314
I20260806 22:43:48.894205 31711 tablet_service.cc:1514] Processing DeleteTablet for tablet 501195803f6848dca02dfed17a624055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:48 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:56876
I20260806 22:43:48.894251 31433 tablet_service.cc:1514] Processing DeleteTablet for tablet 501195803f6848dca02dfed17a624055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:48 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:35330
I20260806 22:43:48.894645 31823 tablet_replica.cc:330] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4: stopping tablet replica
I20260806 22:43:48.894687 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 31371
I20260806 22:43:48.894748 31823 raft_consensus.cc:2229] T 501195803f6848dca02dfed17a624055 P b234a604e19546caad6e5d9b5e4c38e4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:43:48.898751 31821 tablet_replica.cc:330] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: stopping tablet replica
I20260806 22:43:48.898919 31821 raft_consensus.cc:2229] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:48.899068 31821 raft_consensus.cc:2258] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:48.899343 31822 tablet_replica.cc:330] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: stopping tablet replica
I20260806 22:43:48.899479 31822 raft_consensus.cc:2229] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:48.899618 31822 raft_consensus.cc:2258] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:48.899986 31821 ts_tablet_manager.cc:1905] T 501195803f6848dca02dfed17a624055 P c3d3bc553e4d4d73bcc834c8c75d1ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:43:48.901830 31275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:45137 (error 108)
I20260806 22:43:48.902093 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 31508
I20260806 22:43:48.907042 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.906904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:45137 (local address 127.0.0.1:35348)
0806 22:43:48.906951 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.906994 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 22:43:48.907186 31275 catalog_manager.cc:4584] TS b234a604e19546caad6e5d9b5e4c38e4 (127.19.74.193:45137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 501195803f6848dca02dfed17a624055: Network error: Client connection negotiation failed: client connection to 127.19.74.193:45137: connect: Connection refused (error 111)
I20260806 22:43:48.907935 31822 ts_tablet_manager.cc:1905] T 501195803f6848dca02dfed17a624055 P 673747cbe6aa465b9c2ff63a3c2574eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:48.908883 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 31647
I20260806 22:43:48.911106 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.911005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:38213 (local address 127.0.0.1:38316)
0806 22:43:48.911051 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.911075 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.194:38213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:43:48.918008 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 31256
I20260806 22:43:48.918097 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.917998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43675 (local address 127.0.0.1:56890)
0806 22:43:48.918040 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.918062 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.195:43675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 22:43:48.925927 19755 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:43:49.280495 19755 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:43:49.280642 19755 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:43:49.624819 19755 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:43:49.625 UTC [30838] LOG: received smart shutdown request
2026-08-06 22:43:49.628 UTC [30838] LOG: background worker "logical replication launcher" (PID 30847) exited with exit code 1
2026-08-06 22:43:49.629 UTC [30842] LOG: shutting down
2026-08-06 22:43:49.793 UTC [30838] LOG: database system is shut down
2026-08-06T22:43:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46323 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:43:49 dist-test-slave-pgkr krb5kdc[31833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:43:49 dist-test-slave-pgkr krb5kdc[31833](info): set up 2 sockets
Aug 06 22:43:49 dist-test-slave-pgkr krb5kdc[31833](info): commencing operation
krb5kdc: starting...
W20260806 22:43:50.965698 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:43:51 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:43:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.74.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.74.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/rangeradmin_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/rangeradmin_127.19.74.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.74.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.74.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/rangerlookup_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/rangerlookup_127.19.74.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.74.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.74.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/HTTP_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/HTTP_127.19.74.212@KRBTEST.COM.keytab.
I20260806 22:43:51.130975 19755 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLdXImp/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/postgres -l logfile start
2026-08-06 22:43:53.474 UTC [31863] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:43:53.474 UTC [31863] LOG: listening on IPv4 address "127.19.74.212", port 55243
2026-08-06 22:43:53.477 UTC [31863] LOG: listening on Unix socket "/tmp/.s.PGSQL.55243"
2026-08-06 22:43:53.513 UTC [31866] LOG: database system was shut down at 2026-08-06 22:43:52 UTC
2026-08-06 22:43:53.517 UTC [31863] LOG: database system is ready to accept connections
I20260806 22:43:54.479525 19755 mini_postgres.cc:96] Postgres bound to 55243
2026-08-06 22:43:54.486 UTC [31874] FATAL: database "slave" does not exist
127.19.74.212:55243 - accepting connections
I20260806 22:43:54.487448 19755 mini_ranger.cc:161] Starting Ranger...
I20260806 22:43:54.502432 19755 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:43:54.936836 19755 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:43:54.936906 19755 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-admin
2026-08-06 22:43:55,221 [I] DB FLAVOR :POSTGRES
2026-08-06 22:43:55,221 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:43:55,221 [I] Checking connection..
2026-08-06 22:43:55,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:43:55,462 [I] Checking connection passed.
2026-08-06 22:43:55,463 [I] --------- Verifying version history table ---------
2026-08-06 22:43:55,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:43:55,727 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:43:55,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:43:55,969 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:43:55,969 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:43:55,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:43:56.222 UTC [31939] WARNING: there is no transaction in progress
2026-08-06 22:43:56,229 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:43:56,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:43:56,485 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:43:56,485 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:43:56,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:43:56,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:43:57,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pgkr.c.gcp-upstream.internal','N') ;"
2026-08-06 22:43:57,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:43:57,466 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:43:57,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:43:57,720 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:43:57,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:43:57,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:43:58,240 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:43:58,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:43:59.131 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.147 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.161 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.175 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.190 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.206 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.260 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.266 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.279 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.289 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.298 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.311 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.319 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.325 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.331 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59.337 UTC [32050] WARNING: there is no transaction in progress
2026-08-06 22:43:59,899 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:43:59,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:44:00,143 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:44:00,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:44:00,391 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:44:00,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:44:00,654 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:44:00,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:44:00,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:44:01,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pgkr.c.gcp-upstream.internal';"
2026-08-06 22:44:01,359 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:44:01,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:44:01,601 [I] DB_PATCHES have already been applied
I20260806 22:44:01.607129 19755 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLdXImp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:44:01.607190 19755 mini_ranger.cc:193] Using host: 127.19.74.212
I20260806 22:44:01.611153 19755 mini_ranger.cc:227] Ranger admin URL: http://127.19.74.212:59291
Aug 06, 2026 10:44:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:44:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:44:02.833213 19755 mini_ranger.cc:160] Time spent starting Ranger: real 8.346s user 0.002s sys 0.008s
Aug 06, 2026 10:44:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:44:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:44:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:44:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:44:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:44:24.383642 19755 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.74.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.74.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/rangerkms_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.74.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/rangerkms_127.19.74.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.74.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.74.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.74.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/HTTP_127.19.74.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.74.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/HTTP_127.19.74.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:44:24 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056264, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:44:24.466665 19755 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:44:24.476346 19755 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:44:25.224671 19755 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:44:25.224742 19755 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms
I20260806 22:44:25.224756 19755 mini_ranger_kms.cc:227] Using postgres at 127.19.74.212:55243
2026-08-06 22:44:25,395 [I] DB FLAVOR :POSTGRES
2026-08-06 22:44:25,395 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:44:25,395 [I] Checking connection
2026-08-06 22:44:25,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:44:25,635 [I] connection success
2026-08-06 22:44:25,635 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:44:25,635 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:44:25,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:44:25,862 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:44:25,862 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:44:25,863 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:44:25,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.74.212:55243/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:44:26,108 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:44:26.593256 19755 mini_ranger_kms.cc:310] Created kms service
I20260806 22:44:26.789724 19755 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:44:26.834044 19755 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:44:26 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056264, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:27.199839 19755 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:44:27.200083 19755 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLdXImp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLdXImp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:44:27.200110 19755 mini_ranger_kms.cc:240] Using host: 127.19.74.212
I20260806 22:44:27.204739 19755 mini_ranger_kms.cc:276] Ranger KMS PID: 32231
I20260806 22:44:27.204814 19755 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.74.212:46921
Aug 06, 2026 10:44:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:44:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:44:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:44:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:44:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:44:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 22:44:29.519937 19755 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.053s user 0.007s sys 0.147s
I20260806 22:44:29.520058 19755 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:44:29.520105 19755 mini_ranger_kms.cc:401] 127.19.74.212:46921/kms/v1/keys
Aug 06 22:44:29 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056269, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.74.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 22:44:30 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056269, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.74.212@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
Aug 06 22:44:30 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056269, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.74.212@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
Aug 06 22:44:32 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:44:32.649521 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37519
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:37519
--ranger_config_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46921/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:32.731949 32282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:32.732180 32282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:32.732214 32282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:32.733387 32282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:32.733461 32282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:32.733481 32282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:32.733500 32282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:32.733526 32282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:32.735272 32282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.74.212:46921/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:37519
--ranger_config_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:32.735569 32282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:32.735800 32282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:44:32.742137 32282 server_base.cc:1034] running on GCE node
W20260806 22:44:32.742244 32287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:32.742163 32288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:32.742069 32291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:32.742656 32282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:32.742990 32282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:32.744160 32282 hybrid_clock.cc:648] HybridClock initialized: now 1786056272744148 us; error 28 us; skew 500 ppm
Aug 06 22:44:32 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056272, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:32.751658 32282 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:44:32.752214 32282 webserver.cc:460] Webserver started at http://127.19.74.254:40403/ using document root <none> and password file <none>
I20260806 22:44:32.752498 32282 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:32.752558 32282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:32.752697 32282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:44:32 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056272, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:32.804291 32282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "4512395368ec412bb697f8dc925ace88"
format_stamp: "Formatted at 2026-08-06 22:44:32 on dist-test-slave-pgkr"
server_key: "9cc16e1d2578a63ef92a46b4d136de03"
server_key_iv: "e55e2e2fae6c7c3bae0db47a8adaa53c"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:32.804875 32282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "4512395368ec412bb697f8dc925ace88"
format_stamp: "Formatted at 2026-08-06 22:44:32 on dist-test-slave-pgkr"
server_key: "9cc16e1d2578a63ef92a46b4d136de03"
server_key_iv: "e55e2e2fae6c7c3bae0db47a8adaa53c"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:32.807005 32282 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:32.822266 32299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:32.822556 32282 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:44:32.822692 32282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "4512395368ec412bb697f8dc925ace88"
format_stamp: "Formatted at 2026-08-06 22:44:32 on dist-test-slave-pgkr"
server_key: "9cc16e1d2578a63ef92a46b4d136de03"
server_key_iv: "e55e2e2fae6c7c3bae0db47a8adaa53c"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:32.822798 32282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:32.834308 32282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:32.836249 32282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:32.836423 32282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:32.841388 32282 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:37519
I20260806 22:44:32.841464 32352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:37519 every 8 connection(s)
I20260806 22:44:32.841800 32282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:32.842624 32353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:32.845677 32353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Bootstrap starting.
I20260806 22:44:32.846441 32353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:32.846827 32353 log.cc:824] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:32.847371 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 32282
I20260806 22:44:32.847515 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:44:32.847631 32353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: No bootstrap required, opened a new log
I20260806 22:44:32.847704 19755 external_mini_cluster.cc:1428] Setting key b6eb44370f528c14d3006c9efb1cf429
I20260806 22:44:32.850814 32353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } }
I20260806 22:44:32.850980 32353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:32.851064 32353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4512395368ec412bb697f8dc925ace88, State: Initialized, Role: FOLLOWER
I20260806 22:44:32.851256 32353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } }
I20260806 22:44:32.851348 32353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:32.851414 32353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:32.851470 32353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:32.852149 32353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } }
I20260806 22:44:32.852281 32353 leader_election.cc:302] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4512395368ec412bb697f8dc925ace88; no voters:
I20260806 22:44:32.852485 32353 leader_election.cc:288] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:32.852566 32358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:32.852784 32358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 1 LEADER]: Becoming Leader. State: Replica: 4512395368ec412bb697f8dc925ace88, State: Running, Role: LEADER
I20260806 22:44:32.852891 32353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:32.852928 32358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } }
I20260806 22:44:32.853364 32358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4512395368ec412bb697f8dc925ace88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } } }
I20260806 22:44:32.853513 32358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:32.853734 32358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4512395368ec412bb697f8dc925ace88. Latest consensus state: current_term: 1 leader_uuid: "4512395368ec412bb697f8dc925ace88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } } }
I20260806 22:44:32.853806 32358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: This master's current role is: LEADER
Aug 06 22:44:32 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:32.855275 32353 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:44:32.858013 32362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:32.858880 32362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:32.862794 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:32.848204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44880 (local address 127.19.74.254:37519)
0806 22:44:32.848393 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:32.848403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:32.848438 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:44:32.849216 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:32.849225 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:32.849270 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:32.849407 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:32.850029 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:32.850967 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:32.851894 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:32.852426 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:32.855525 (+ 3099us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:32.855669 (+ 144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:32.855672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:32.855673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:32.855683 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:32.860637 (+ 4954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:32.861077 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:32.861080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:32.861080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:32.861179 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:32.861400 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:32.861400 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:32.861401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:32.861725 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:32.862301 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:32.862378 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:44:32.862523 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260806 22:44:32.865875 32362 catalog_manager.cc:1347] Generated new cluster ID: d6d1b37f1fbf4a969afe610bd1036f0d
I20260806 22:44:32.865919 32362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:32.897291 32362 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:32.898166 32362 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:32.902565 32362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Generated new TSK 0
I20260806 22:44:32.902829 32362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:44:33.669607 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37519
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:33.792299 32398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:33.792572 32398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:33.792608 32398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:33.794265 32398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:33.794355 32398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:33.794392 32398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:44:33.796461 32398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.74.212:46921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:33.796706 32398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:33.796996 32398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:33.803094 32404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:33.803107 32406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:33.803617 32403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:33.804136 32398 server_base.cc:1034] running on GCE node
I20260806 22:44:33.804344 32398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:33.804633 32398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:33.805799 32398 hybrid_clock.cc:648] HybridClock initialized: now 1786056273805768 us; error 43 us; skew 500 ppm
Aug 06 22:44:33 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056273, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:33.809198 32398 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:44:33.809768 32398 webserver.cc:460] Webserver started at http://127.19.74.193:33919/ using document root <none> and password file <none>
I20260806 22:44:33.810043 32398 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:33.810122 32398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:33.810276 32398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:44:33 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056273, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:33.823207 32398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "34820d2299204f0c9600100869c0adfc"
format_stamp: "Formatted at 2026-08-06 22:44:33 on dist-test-slave-pgkr"
server_key: "48ae5ecc7bb69d2508c095db587e9413"
server_key_iv: "46a4c893d03f56b7c6f5cb1e702177a9"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:33.823730 32398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "34820d2299204f0c9600100869c0adfc"
format_stamp: "Formatted at 2026-08-06 22:44:33 on dist-test-slave-pgkr"
server_key: "48ae5ecc7bb69d2508c095db587e9413"
server_key_iv: "46a4c893d03f56b7c6f5cb1e702177a9"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:33.825896 32398 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:33.839522 32413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:33.839758 32398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:33.839869 32398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "34820d2299204f0c9600100869c0adfc"
format_stamp: "Formatted at 2026-08-06 22:44:33 on dist-test-slave-pgkr"
server_key: "48ae5ecc7bb69d2508c095db587e9413"
server_key_iv: "46a4c893d03f56b7c6f5cb1e702177a9"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:33.839957 32398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:33.868660 32398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:33.871162 32398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:33.871346 32398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:33.871665 32398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:33.872081 32398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:33.872138 32398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:33.872174 32398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:33.872272 32398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:33.880177 32398 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:43583
I20260806 22:44:33.880263 32530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:43583 every 8 connection(s)
I20260806 22:44:33.880625 32398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:33.887802 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 32398
I20260806 22:44:33.887909 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:33.888106 19755 external_mini_cluster.cc:1428] Setting key 628474e6519cb70f22eabff17254be39
Aug 06 22:44:33 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056273, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:33.893471 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:33.881480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37519 (local address 127.19.74.193:58001)
0806 22:44:33.881691 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:33.881739 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:33.882287 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:33.882585 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:33.882596 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:33.882846 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:33.883062 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:33.883067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:33.883797 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:33.883799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:33.884596 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:33.884598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:33.885069 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:33.885070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:33.887068 (+ 1998us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:33.887680 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:33.887692 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:33.889475 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:33.891687 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:33.891695 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:33.891697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:33.892002 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:33.892254 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:33.892255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:33.892255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:33.892399 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:33.892750 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:33.892758 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:33.892925 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:44:33.893045 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:44:33.893220 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260806 22:44:33.894028 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:33.881514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:58001 (local address 127.19.74.254:37519)
0806 22:44:33.881681 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:33.881683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:33.881834 (+ 151us) server_negotiation.cc:407] Connection header received
0806 22:44:33.882400 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:33.882401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:33.882416 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:33.882487 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:33.883171 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:33.883698 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:33.884695 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:33.884960 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:33.885046 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:33.889583 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:33.889585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:33.889585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:33.889593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:33.891582 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:33.892094 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:33.892095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:33.892096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:33.892159 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:33.892480 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:33.892481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:33.892481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:33.892652 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:33.892719 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:33.893841 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0806 22:44:33.893935 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260806 22:44:33.894896 32531 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37519
I20260806 22:44:33.894995 32531 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:33.895193 32531 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:33.895849 32317 ts_manager.cc:194] Registered new tserver with Master: 34820d2299204f0c9600100869c0adfc (127.19.74.193:43583)
I20260806 22:44:33.896778 32317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:58001
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:44:33.918799 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37519
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:44:34.044541 32537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:34.044874 32537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:34.044958 32537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:34.046705 32537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:34.046792 32537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:34.046860 32537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:44:34.048770 32537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.74.212:46921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:34.049037 32537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:34.049276 32537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:34.055128 32545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:34.055737 32547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:34.056557 32544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:34.057054 32537 server_base.cc:1034] running on GCE node
I20260806 22:44:34.057300 32537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:34.057622 32537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:34.058794 32537 hybrid_clock.cc:648] HybridClock initialized: now 1786056274058770 us; error 42 us; skew 500 ppm
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:34.061865 32537 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:44:34.062397 32537 webserver.cc:460] Webserver started at http://127.19.74.194:40863/ using document root <none> and password file <none>
I20260806 22:44:34.062713 32537 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:34.062774 32537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:34.062906 32537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:34.077997 32537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "ae02d9a743b94dedb7177540ab4ae8c4"
format_stamp: "Formatted at 2026-08-06 22:44:34 on dist-test-slave-pgkr"
server_key: "e51f843f0e1292c9f4a4f79270cbb248"
server_key_iv: "ad537d24e2568b15ddc50c0b9dca23a5"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:34.078496 32537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "ae02d9a743b94dedb7177540ab4ae8c4"
format_stamp: "Formatted at 2026-08-06 22:44:34 on dist-test-slave-pgkr"
server_key: "e51f843f0e1292c9f4a4f79270cbb248"
server_key_iv: "ad537d24e2568b15ddc50c0b9dca23a5"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:34.086333 32537 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.001s
I20260806 22:44:34.102375 32554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.102658 32537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 22:44:34.102777 32537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "ae02d9a743b94dedb7177540ab4ae8c4"
format_stamp: "Formatted at 2026-08-06 22:44:34 on dist-test-slave-pgkr"
server_key: "e51f843f0e1292c9f4a4f79270cbb248"
server_key_iv: "ad537d24e2568b15ddc50c0b9dca23a5"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:34.102873 32537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:34.126520 32537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:34.129069 32537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:34.129273 32537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:34.129540 32537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:34.129957 32537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:34.130048 32537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.130136 32537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:34.130195 32537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.137360 32537 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:46603
I20260806 22:44:34.137858 32537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:34.140762 32671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:46603 every 8 connection(s)
I20260806 22:44:34.140985 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 32537
I20260806 22:44:34.141091 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:34.141306 19755 external_mini_cluster.cc:1428] Setting key cf35ae152438b8e3de8eddb85ae19862
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:34.152601 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:34.141422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:50853 (local address 127.19.74.254:37519)
0806 22:44:34.141476 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:34.141479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:34.142004 (+ 525us) server_negotiation.cc:407] Connection header received
0806 22:44:34.142737 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:34.142739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:34.142755 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:34.142784 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:34.143650 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.144392 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.145830 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.146125 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.146170 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:34.148912 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:34.148913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:34.148913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:34.148921 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:34.150838 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:34.151396 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:34.151397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:34.151398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:34.151463 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:34.151778 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:34.151779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:34.151779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:34.151926 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:34.151969 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:34.152380 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 22:44:34.152474 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
I20260806 22:44:34.153399 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:34.141418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37519 (local address 127.19.74.194:50853)
0806 22:44:34.141868 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:34.141916 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:34.142595 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:34.142975 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:34.142980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:34.143228 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:34.143533 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:34.143541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.144528 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.144529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:34.145728 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:34.145731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.146295 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.146297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:34.146456 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:34.147097 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:34.147108 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:34.148801 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:34.150936 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:34.150943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:34.150946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:34.151266 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:34.151540 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:34.151541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:34.151542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:34.151689 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:34.152020 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:34.152026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:34.152207 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 22:44:34.152976 (+ 769us) client_negotiation.cc:240] Negotiation successful
0806 22:44:34.153161 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":108,"threads_started":1}
I20260806 22:44:34.154150 32672 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37519
I20260806 22:44:34.154263 32672 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:34.154466 32672 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:34.154930 32317 ts_manager.cc:194] Registered new tserver with Master: ae02d9a743b94dedb7177540ab4ae8c4 (127.19.74.194:46603)
I20260806 22:44:34.155419 32317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:50853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:44:34.175945 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37519
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:34.303571 32677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:34.303920 32677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:34.303966 32677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:34.305696 32677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:34.305756 32677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:34.305795 32677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:44:34.307942 32677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.74.212:46921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:34.308350 32677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:34.308673 32677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:34.315412 32685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:34.316021 32683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:34.316128 32682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:34.317309 32677 server_base.cc:1034] running on GCE node
I20260806 22:44:34.317540 32677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:34.317829 32677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:34.318992 32677 hybrid_clock.cc:648] HybridClock initialized: now 1786056274318961 us; error 38 us; skew 500 ppm
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:34.322439 32677 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:44:34.323037 32677 webserver.cc:460] Webserver started at http://127.19.74.195:34269/ using document root <none> and password file <none>
I20260806 22:44:34.323333 32677 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:34.323414 32677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:34.323570 32677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:34.338055 32677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "9b2a66da19db4467964b03d9b0c6723f"
format_stamp: "Formatted at 2026-08-06 22:44:34 on dist-test-slave-pgkr"
server_key: "c4ebc1052fe5e37eecd156232cb4d305"
server_key_iv: "758b4ee4796cb1cf6aea8bb4d8de73a0"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:34.338686 32677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "9b2a66da19db4467964b03d9b0c6723f"
format_stamp: "Formatted at 2026-08-06 22:44:34 on dist-test-slave-pgkr"
server_key: "c4ebc1052fe5e37eecd156232cb4d305"
server_key_iv: "758b4ee4796cb1cf6aea8bb4d8de73a0"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:34.345156 32677 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260806 22:44:34.359081 32692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.359318 32677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:34.359426 32677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "9b2a66da19db4467964b03d9b0c6723f"
format_stamp: "Formatted at 2026-08-06 22:44:34 on dist-test-slave-pgkr"
server_key: "c4ebc1052fe5e37eecd156232cb4d305"
server_key_iv: "758b4ee4796cb1cf6aea8bb4d8de73a0"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:34.359536 32677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:34.387102 32677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:34.389799 32677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:34.389998 32677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:34.390309 32677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:34.390725 32677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:34.390808 32677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.390874 32677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:34.390931 32677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.401188 32677 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:35975
I20260806 22:44:34.401736 32677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:34.403672 338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:35975 every 8 connection(s)
I20260806 22:44:34.407255 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 32677
I20260806 22:44:34.407387 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:34.407618 19755 external_mini_cluster.cc:1428] Setting key eec1eb2f05cfc954c6fb7c09069ef92f
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:34.416929 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:34.404381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:60331 (local address 127.19.74.254:37519)
0806 22:44:34.404433 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:34.404435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:34.405047 (+ 612us) server_negotiation.cc:407] Connection header received
0806 22:44:34.405756 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:34.405758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:34.405771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:34.405848 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:34.408548 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.409362 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.410767 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.411048 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.411136 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:34.413763 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:34.413764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:34.413764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:34.413771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:34.415407 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:34.415882 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:34.415883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:34.415883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:34.415941 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:34.416189 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:34.416190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:34.416190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:34.416314 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:34.416353 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:34.416746 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 22:44:34.416830 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:44:34.417678 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:34.404368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37519 (local address 127.19.74.195:60331)
0806 22:44:34.404911 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:34.404951 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:34.405639 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:34.406980 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:34.406986 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:34.407244 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:34.407521 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:34.407528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.409483 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.409484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:34.410654 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:34.410658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.411168 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.411169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:34.411302 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:34.411965 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:34.411976 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:34.413656 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:34.415500 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:34.415507 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:34.415509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:34.415796 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:34.416006 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:34.416007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:34.416007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:34.416122 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:34.416394 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:34.416399 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:34.416573 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 22:44:34.417268 (+ 695us) client_negotiation.cc:240] Negotiation successful
0806 22:44:34.417427 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":119,"threads_started":1}
I20260806 22:44:34.418303 340 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37519
I20260806 22:44:34.418443 340 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:34.418704 340 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:34.419131 32316 ts_manager.cc:194] Registered new tserver with Master: 9b2a66da19db4467964b03d9b0c6723f (127.19.74.195:35975)
I20260806 22:44:34.419641 32316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:60331
I20260806 22:44:34.420101 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:44:34.420742 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 32398
I20260806 22:44:34.429764 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 32537
I20260806 22:44:34.436479 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 32677
I20260806 22:44:34.443522 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 32282
I20260806 22:44:34.452417 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37519
--webserver_interface=127.19.74.254
--webserver_port=40403
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:37519
--ranger_config_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46921/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.74.212:46921/kms with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:34.537915 343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:34.538156 343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:34.538179 343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:34.539426 343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:34.539477 343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:34.539498 343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:34.539515 343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:34.539552 343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:34.541191 343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.74.212:46921/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:37519
--ranger_config_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=40403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:34.541447 343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:34.541708 343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:34.546801 349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:34.546922 351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:34.546828 348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:34.547199 343 server_base.cc:1034] running on GCE node
I20260806 22:44:34.547428 343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:34.547742 343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:34.548899 343 hybrid_clock.cc:648] HybridClock initialized: now 1786056274548881 us; error 34 us; skew 500 ppm
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:34.551518 343 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:44:34.552189 343 webserver.cc:460] Webserver started at http://127.19.74.254:40403/ using document root <none> and password file <none>
I20260806 22:44:34.552531 343 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:34.552614 343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:34.600528 343 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:34.601423 359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.601672 343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:34.601787 343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "4512395368ec412bb697f8dc925ace88"
format_stamp: "Formatted at 2026-08-06 22:44:32 on dist-test-slave-pgkr"
server_key: "9cc16e1d2578a63ef92a46b4d136de03"
server_key_iv: "e55e2e2fae6c7c3bae0db47a8adaa53c"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:34.602201 343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:34.618386 343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:34.620738 343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:34.620929 343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:34.626332 343 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:37519
I20260806 22:44:34.626366 411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:37519 every 8 connection(s)
I20260806 22:44:34.626848 343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:34.628146 412 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:44:34.628854 412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Bootstrap starting.
I20260806 22:44:34.629434 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 343
I20260806 22:44:34.629786 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:43583
--local_ip_for_outbound_sockets=127.19.74.193
--tserver_master_addrs=127.19.74.254:37519
--webserver_port=33919
--webserver_interface=127.19.74.193
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.74.212:46921/kms with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:34.639432 412 log.cc:824] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:34.641929 412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:44:34.642303 412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Bootstrap complete.
I20260806 22:44:34.645975 412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } }
I20260806 22:44:34.646319 412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4512395368ec412bb697f8dc925ace88, State: Initialized, Role: FOLLOWER
I20260806 22:44:34.646519 412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } }
I20260806 22:44:34.646595 412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:34.646672 412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:34.646728 412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:44:34.648087 412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } }
I20260806 22:44:34.648209 412 leader_election.cc:302] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4512395368ec412bb697f8dc925ace88; no voters:
I20260806 22:44:34.648377 412 leader_election.cc:288] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:44:34.648458 416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:44:34.648650 416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [term 2 LEADER]: Becoming Leader. State: Replica: 4512395368ec412bb697f8dc925ace88, State: Running, Role: LEADER
I20260806 22:44:34.648653 412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:34.648782 416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } }
I20260806 22:44:34.649019 416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4512395368ec412bb697f8dc925ace88. Latest consensus state: current_term: 2 leader_uuid: "4512395368ec412bb697f8dc925ace88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } } }
I20260806 22:44:34.649020 417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4512395368ec412bb697f8dc925ace88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4512395368ec412bb697f8dc925ace88" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37519 } } }
I20260806 22:44:34.649214 416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:34.649217 417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:34.649753 425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:34.650264 425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:34.650414 425 catalog_manager.cc:1259] Loaded cluster ID: d6d1b37f1fbf4a969afe610bd1036f0d
I20260806 22:44:34.650445 425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:34.650596 412 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:44:34.651465 425 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:34.651612 425 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Loaded TSK: 0
I20260806 22:44:34.651751 425 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:44:34.747427 414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:34.747721 414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:34.747758 414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:34.749387 414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:34.749513 414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:34.749605 414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:44:34.751643 414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.74.212:46921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:43583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=33919
--tserver_master_addrs=127.19.74.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:34.751870 414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:34.752185 414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:34.758883 449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:34.759541 414 server_base.cc:1034] running on GCE node
W20260806 22:44:34.759742 452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:34.760015 450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:34.760313 414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:34.760578 414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:34.761756 414 hybrid_clock.cc:648] HybridClock initialized: now 1786056274761739 us; error 30 us; skew 500 ppm
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:34.765072 414 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:44:34.765623 414 webserver.cc:460] Webserver started at http://127.19.74.193:33919/ using document root <none> and password file <none>
I20260806 22:44:34.765875 414 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:34.765961 414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:34.790011 414 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260806 22:44:34.801687 461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.801945 414 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.001s sys 0.000s
I20260806 22:44:34.802037 414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "34820d2299204f0c9600100869c0adfc"
format_stamp: "Formatted at 2026-08-06 22:44:33 on dist-test-slave-pgkr"
server_key: "48ae5ecc7bb69d2508c095db587e9413"
server_key_iv: "46a4c893d03f56b7c6f5cb1e702177a9"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:34.802505 414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:34.825906 414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:34.828814 414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:34.829043 414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:34.829312 414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:34.829715 414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:34.829798 414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.829862 414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:34.829892 414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:34.836684 414 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:43583
I20260806 22:44:34.837217 414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:34.841634 576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:43583 every 8 connection(s)
I20260806 22:44:34.846000 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 414
I20260806 22:44:34.846338 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:46603
--local_ip_for_outbound_sockets=127.19.74.194
--tserver_master_addrs=127.19.74.254:37519
--webserver_port=40863
--webserver_interface=127.19.74.194
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.74.212:46921/kms with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:34.855208 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:34.840813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37519 (local address 127.19.74.193:50027)
0806 22:44:34.842724 (+ 1911us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:34.842772 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:34.843339 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:34.844049 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:34.844052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:34.844267 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:34.844521 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:34.844527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.846155 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.846156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:34.847595 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:34.847598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.848679 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.848680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:34.848809 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:34.849446 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:34.849457 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:34.851230 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:34.853417 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:34.853423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:34.853425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:34.853706 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:34.853948 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:34.853949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:34.853949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:34.854051 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:34.854472 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:34.854477 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:34.854665 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 22:44:34.854784 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:44:34.854950 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1865,"thread_start_us":84,"threads_started":1}
I20260806 22:44:34.856052 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:34.841157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:50027 (local address 127.19.74.254:37519)
0806 22:44:34.843530 (+ 2373us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:34.843539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:34.843583 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:44:34.843719 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:34.843726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:34.843775 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:34.843884 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:34.844660 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.845996 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.847719 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:34.848427 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:34.848534 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:34.851361 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:34.851366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:34.851367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:34.851381 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:34.853324 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:34.853791 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:34.853794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:34.853795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:34.853885 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:34.854114 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:34.854115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:34.854115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:34.854380 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:34.854447 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:34.855704 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0806 22:44:34.855835 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2310,"thread_start_us":67,"threads_started":1}
I20260806 22:44:34.857050 577 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37519
I20260806 22:44:34.857149 577 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:34.857327 577 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
W20260806 22:44:34.858182 577 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:37519 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:44:34.858244 577 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:34.858300 577 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:34.858697 577 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:34.858772 577 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
W20260806 22:44:34.859071 577 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:44:34.977032 581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:34.977300 581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:34.977336 581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:34.979034 581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:34.979094 581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:34.979136 581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:44:34.981367 581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.74.212:46921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:46603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=40863
--tserver_master_addrs=127.19.74.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:34.981632 581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:34.981938 581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:34.987694 586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:34.987815 581 server_base.cc:1034] running on GCE node
W20260806 22:44:34.987984 589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:34.988152 587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:34.988400 581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:34.988675 581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:34.989861 581 hybrid_clock.cc:648] HybridClock initialized: now 1786056274989829 us; error 44 us; skew 500 ppm
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:34.993191 581 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:44:34.993777 581 webserver.cc:460] Webserver started at http://127.19.74.194:40863/ using document root <none> and password file <none>
I20260806 22:44:34.994066 581 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:34.994154 581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:44:34 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:35.013216 581 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:35.014238 597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:35.014433 581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:35.014536 581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "ae02d9a743b94dedb7177540ab4ae8c4"
format_stamp: "Formatted at 2026-08-06 22:44:34 on dist-test-slave-pgkr"
server_key: "e51f843f0e1292c9f4a4f79270cbb248"
server_key_iv: "ad537d24e2568b15ddc50c0b9dca23a5"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:35.014999 581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:35.045691 581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:35.047964 581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:35.048146 581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:35.048452 581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:35.048807 581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:35.048893 581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:35.048964 581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:35.049000 581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:35.055492 581 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:46603
I20260806 22:44:35.055622 712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:46603 every 8 connection(s)
I20260806 22:44:35.055969 581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:44:35 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:35.063699 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 581
I20260806 22:44:35.064016 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.74.212:46921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:35975
--local_ip_for_outbound_sockets=127.19.74.195
--tserver_master_addrs=127.19.74.254:37519
--webserver_port=34269
--webserver_interface=127.19.74.195
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.74.212:46921/kms with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:35.067471 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:35.056754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:43327 (local address 127.19.74.254:37519)
0806 22:44:35.056813 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:35.056816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:35.057085 (+ 269us) server_negotiation.cc:407] Connection header received
0806 22:44:35.057732 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:35.057733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:35.057744 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:35.057797 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:35.058545 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.059363 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.060897 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.061225 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.061304 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:35.063944 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:35.063946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:35.063946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:35.063954 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:35.065734 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:35.066273 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:35.066274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:35.066275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:35.066341 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:35.066676 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:35.066678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:35.066678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:35.066821 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:35.066867 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:35.067274 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 22:44:35.067373 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:44:35.068315 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:35.056733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37519 (local address 127.19.74.194:43327)
0806 22:44:35.056911 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:35.056958 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:35.057600 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:35.057882 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:35.057887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:35.058137 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:35.058421 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:35.058430 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.059490 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.059491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:35.060792 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:35.060795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.061350 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.061351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:35.061496 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:35.062083 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:35.062093 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:35.063831 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:35.065850 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:35.065858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:35.065860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:35.066172 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:35.066424 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:35.066424 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:35.066425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:35.066556 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:35.066919 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:35.066925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:35.067095 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:44:35.067906 (+ 811us) client_negotiation.cc:240] Negotiation successful
0806 22:44:35.068081 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260806 22:44:35.069061 713 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37519
I20260806 22:44:35.069161 713 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:35.069344 713 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
W20260806 22:44:35.070050 713 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:37519 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:44:35.070123 713 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:35.070184 713 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:35.070540 713 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:35.070642 713 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
W20260806 22:44:35.071049 713 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:44:35.187340 716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:35.187733 716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:35.187824 716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:35.189497 716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:35.189610 716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:35.189697 716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:44:35.192098 716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.74.212:46921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:35975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=34269
--tserver_master_addrs=127.19.74.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:35.192637 716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:35.192953 716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:35.202852 722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:35.202891 725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:35.203328 723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:35.203663 716 server_base.cc:1034] running on GCE node
I20260806 22:44:35.203888 716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:35.204205 716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:35.219666 716 hybrid_clock.cc:648] HybridClock initialized: now 1786056275219656 us; error 43 us; skew 500 ppm
Aug 06 22:44:35 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:35.222575 716 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:44:35.223196 716 webserver.cc:460] Webserver started at http://127.19.74.195:34269/ using document root <none> and password file <none>
I20260806 22:44:35.223449 716 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:35.223508 716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:44:35 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:35.240221 716 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:35.241272 733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:35.241461 716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:44:35.241544 716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "9b2a66da19db4467964b03d9b0c6723f"
format_stamp: "Formatted at 2026-08-06 22:44:34 on dist-test-slave-pgkr"
server_key: "c4ebc1052fe5e37eecd156232cb4d305"
server_key_iv: "758b4ee4796cb1cf6aea8bb4d8de73a0"
server_key_version: "kuduclusterkey@0"
I20260806 22:44:35.241992 716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:35.267248 716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:35.269834 716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:35.270102 716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:35.270442 716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:35.270886 716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:35.270938 716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:35.270967 716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:35.270984 716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:35.278549 716 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:35975
I20260806 22:44:35.278677 849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:35975 every 8 connection(s)
I20260806 22:44:35.279057 716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:44:35 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:35.288743 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 716
I20260806 22:44:35.290484 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:35.280058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:53431 (local address 127.19.74.254:37519)
0806 22:44:35.280154 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:35.280158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:35.280394 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:44:35.281088 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:35.281089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:35.281099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:35.281152 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:35.281825 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.282296 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.283814 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.284169 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.284250 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:35.287135 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:35.287137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:35.287138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:35.287146 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:35.288943 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:35.289450 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:35.289452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:35.289452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:35.289520 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:35.289799 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:35.289800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:35.289800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:35.289923 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:35.289995 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:35.290289 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 22:44:35.290387 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:44:35.290721 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:35.280037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37519 (local address 127.19.74.195:53431)
0806 22:44:35.280244 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:35.280293 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:35.280938 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:35.281229 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:35.281234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:35.281461 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:35.281736 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:35.281743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.282394 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.282395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:35.283706 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:35.283710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.284348 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.284350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:35.284518 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:35.285226 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:35.285254 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:35.286999 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:35.289022 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:35.289029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:35.289031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:35.289346 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:35.289598 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:35.289599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:35.289600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:35.289720 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:35.290000 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:35.290006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:35.290149 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 22:44:35.290255 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:44:35.290406 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
I20260806 22:44:35.291388 850 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37519
I20260806 22:44:35.291503 850 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:35.291718 850 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
W20260806 22:44:35.292447 850 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:37519 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:44:35.292591 850 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:35.292672 850 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:35.293023 850 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:35.293135 850 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
W20260806 22:44:35.293481 850 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:44:35.301450 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:35.289227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47854 (local address 127.19.74.254:37519)
0806 22:44:35.291037 (+ 1810us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:35.291039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:35.291047 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:35.291067 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:35.291068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:35.291084 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:35.291136 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:35.295488 (+ 4352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.295954 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.297070 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:35.297265 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:35.297322 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:35.298833 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:35.298834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:35.298835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:35.298842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:35.300317 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:35.300732 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:35.300733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:35.300733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:35.300783 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:35.301006 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:35.301006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:35.301006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:35.301103 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:35.301216 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:35.301329 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:44:35.301384 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1794,"thread_start_us":72,"threads_started":1}
Aug 06 22:44:35 dist-test-slave-pgkr krb5kdc[31833](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:35.859236 577 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:35.859406 577 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:35.859797 376 ts_manager.cc:194] Registered new tserver with Master: 34820d2299204f0c9600100869c0adfc (127.19.74.193:43583)
I20260806 22:44:35.860831 376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:50027
I20260806 22:44:36.071189 713 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:36.071341 713 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:36.071717 376 ts_manager.cc:194] Registered new tserver with Master: ae02d9a743b94dedb7177540ab4ae8c4 (127.19.74.194:46603)
I20260806 22:44:36.072290 376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:43327
Aug 06 22:44:36 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
I20260806 22:44:36.293712 850 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:36.293895 850 heartbeater.cc:511] Master 127.19.74.254:37519 requested a full tablet report, sending...
I20260806 22:44:36.294167 376 ts_manager.cc:194] Registered new tserver with Master: 9b2a66da19db4467964b03d9b0c6723f (127.19.74.195:35975)
I20260806 22:44:36.294687 376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:53431
I20260806 22:44:36.301299 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:44:36.310971 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.302299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.19.74.254:37519)
0806 22:44:36.302423 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:36.302425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:36.302491 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:44:36.302568 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:36.302569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:36.302582 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:36.302701 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:36.303443 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.304206 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.305326 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.305690 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.305765 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.307561 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:36.307563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:36.307563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:36.307571 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:36.309490 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:36.309965 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:36.309967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:36.309967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:36.310037 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:36.310333 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:36.310334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:36.310334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:36.310473 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:36.310591 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:36.310773 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 22:44:36.310865 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260806 22:44:36.313544 376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:36.315994 376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:36.325598 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.319723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:35975 (local address 127.0.0.1:33970)
0806 22:44:36.320576 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:36.320617 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:36.320681 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:36.321211 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:36.321215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:36.321217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:36.321288 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.321290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.322501 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.322502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.324040 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.324043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.325350 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.325351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.325411 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.325416 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:36.325443 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 22:44:36.325503 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":122,"threads_started":1}
I20260806 22:44:36.325989 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.320210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45574 (local address 127.19.74.193:43583)
0806 22:44:36.320355 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:36.320361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:36.321392 (+ 1031us) server_negotiation.cc:407] Connection header received
0806 22:44:36.321519 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:36.321522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:36.321538 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:36.321596 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:36.321863 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.322944 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.324284 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.325063 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.325122 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.325198 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:36.325520 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 22:44:36.325569 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260806 22:44:36.326448 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.319724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:43583 (local address 127.0.0.1:45574)
0806 22:44:36.321320 (+ 1596us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:36.321331 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:36.321421 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:36.321714 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:36.321715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:36.321717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:36.321774 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.321775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.323046 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.323047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.324161 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.324165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.325270 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.325271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.325366 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.325382 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:44:36.326287 (+ 905us) client_negotiation.cc:240] Negotiation successful
0806 22:44:36.326366 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1574,"mutex_wait_us":5,"thread_start_us":115,"threads_started":1}
I20260806 22:44:36.327452 510 tablet_service.cc:1467] Processing CreateTablet for tablet 752bb645d0ec49108a6c98273a46a7c7 (DEFAULT_TABLE table=test-table [id=225525755f3947bcbf209c9eb469a529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:36.328104 510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752bb645d0ec49108a6c98273a46a7c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:36.330844 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.320424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:46603 (local address 127.0.0.1:35674)
0806 22:44:36.325855 (+ 5431us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:36.325865 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:36.325908 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:36.326178 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:36.326179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:36.326180 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:36.326210 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.326211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.328609 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.328610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.329606 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.329609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.330575 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.330576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.330688 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.330691 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:36.330734 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 22:44:36.330773 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5422,"thread_start_us":85,"threads_started":1}
I20260806 22:44:36.331511 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.320054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33970 (local address 127.19.74.195:35975)
0806 22:44:36.321033 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:36.321038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:36.321050 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:44:36.321087 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:36.321089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:36.321106 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:36.321196 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:36.321376 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.322401 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.324148 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.325264 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.331276 (+ 6012us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.331359 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:36.331388 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:44:36.331436 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":61,"threads_started":1}
I20260806 22:44:36.331766 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.320743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.19.74.194:46603)
0806 22:44:36.320894 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:36.320899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:36.325969 (+ 5070us) server_negotiation.cc:407] Connection header received
0806 22:44:36.326011 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:36.326014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:36.326030 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:36.326102 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:36.326275 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.328509 (+ 2234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.329703 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.330386 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.330467 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.330526 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:36.331651 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0806 22:44:36.331696 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260806 22:44:36.332370 781 tablet_service.cc:1467] Processing CreateTablet for tablet 752bb645d0ec49108a6c98273a46a7c7 (DEFAULT_TABLE table=test-table [id=225525755f3947bcbf209c9eb469a529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:36.332544 646 tablet_service.cc:1467] Processing CreateTablet for tablet 752bb645d0ec49108a6c98273a46a7c7 (DEFAULT_TABLE table=test-table [id=225525755f3947bcbf209c9eb469a529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:36.332757 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752bb645d0ec49108a6c98273a46a7c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:36.332886 646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752bb645d0ec49108a6c98273a46a7c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:36.333649 883 tablet_bootstrap.cc:492] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc: Bootstrap starting.
I20260806 22:44:36.334681 883 tablet_bootstrap.cc:654] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:36.335080 883 log.cc:824] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:36.335927 883 tablet_bootstrap.cc:492] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc: No bootstrap required, opened a new log
I20260806 22:44:36.336001 883 ts_tablet_manager.cc:1397] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:36.336504 885 tablet_bootstrap.cc:492] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Bootstrap starting.
I20260806 22:44:36.337138 885 tablet_bootstrap.cc:654] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:36.337420 885 log.cc:824] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:36.338743 883 raft_consensus.cc:348] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.339010 883 raft_consensus.cc:374] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:36.339118 883 raft_consensus.cc:729] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34820d2299204f0c9600100869c0adfc, State: Initialized, Role: FOLLOWER
I20260806 22:44:36.339123 886 tablet_bootstrap.cc:492] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Bootstrap starting.
I20260806 22:44:36.339332 883 consensus_queue.cc:260] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.339704 886 tablet_bootstrap.cc:654] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:36.339720 883 ts_tablet_manager.cc:1428] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 22:44:36.339932 577 heartbeater.cc:503] Master 127.19.74.254:37519 was elected leader, sending a full tablet report...
I20260806 22:44:36.339986 886 log.cc:824] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:36.340824 885 tablet_bootstrap.cc:492] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: No bootstrap required, opened a new log
I20260806 22:44:36.340878 885 ts_tablet_manager.cc:1397] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
W20260806 22:44:36.342383 578 tablet.cc:2369] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:44:36.343266 885 raft_consensus.cc:348] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.343421 885 raft_consensus.cc:374] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:36.343451 885 raft_consensus.cc:729] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae02d9a743b94dedb7177540ab4ae8c4, State: Initialized, Role: FOLLOWER
I20260806 22:44:36.343577 885 consensus_queue.cc:260] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.343817 885 ts_tablet_manager.cc:1428] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:36.343852 713 heartbeater.cc:503] Master 127.19.74.254:37519 was elected leader, sending a full tablet report...
I20260806 22:44:36.347178 886 tablet_bootstrap.cc:492] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: No bootstrap required, opened a new log
I20260806 22:44:36.347249 886 ts_tablet_manager.cc:1397] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260806 22:44:36.349736 886 raft_consensus.cc:348] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.349900 886 raft_consensus.cc:374] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:36.350097 886 raft_consensus.cc:729] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b2a66da19db4467964b03d9b0c6723f, State: Initialized, Role: FOLLOWER
I20260806 22:44:36.350265 886 consensus_queue.cc:260] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.350525 886 ts_tablet_manager.cc:1428] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:36.350566 850 heartbeater.cc:503] Master 127.19.74.254:37519 was elected leader, sending a full tablet report...
Aug 06 22:44:36 dist-test-slave-pgkr krb5kdc[31833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for HTTP/127.19.74.212@KRBTEST.COM
W20260806 22:44:36.530364 851 tablet.cc:2369] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:44:36.557329 714 tablet.cc:2369] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:44:36.683161 889 raft_consensus.cc:482] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:36.683321 889 raft_consensus.cc:504] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.683771 889 leader_election.cc:288] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b2a66da19db4467964b03d9b0c6723f (127.19.74.195:35975), ae02d9a743b94dedb7177540ab4ae8c4 (127.19.74.194:46603)
I20260806 22:44:36.688177 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.684187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:47615 (local address 127.19.74.195:35975)
0806 22:44:36.684291 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:36.684294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:36.684347 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:44:36.684466 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:36.684468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:36.684480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:36.684529 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:36.684769 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.685548 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.687192 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.687750 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.687801 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.687809 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:36.688072 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 22:44:36.688111 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:44:36.688177 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.684246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:42333 (local address 127.19.74.194:46603)
0806 22:44:36.684327 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:36.684328 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:36.684564 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:44:36.684668 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:36.684669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:36.684679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:36.684711 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:36.684910 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.685597 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.687191 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.687750 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.687801 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.687809 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:36.688060 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:44:36.688103 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:44:36.688423 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.683937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:46603 (local address 127.19.74.193:42333)
0806 22:44:36.684501 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:36.684510 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:36.684589 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:36.684777 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:36.684777 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:36.684778 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:36.684838 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.684839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.685685 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.685685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.687077 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.687081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.687895 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.687896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.687973 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.687976 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:36.688264 (+ 288us) client_negotiation.cc:240] Negotiation successful
0806 22:44:36.688315 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":69,"thread_start_us":130,"threads_started":1}
I20260806 22:44:36.688722 902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.683928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:35975 (local address 127.19.74.193:47615)
0806 22:44:36.684273 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:36.684286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:36.684383 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:36.684587 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:36.684588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:36.684590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:36.684675 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.684678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.685654 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.685655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.687077 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:36.687081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.687895 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.687896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:36.687973 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.687977 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:36.688586 (+ 609us) client_negotiation.cc:240] Negotiation successful
0806 22:44:36.688626 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":81,"thread_start_us":111,"threads_started":1}
I20260806 22:44:36.688966 666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752bb645d0ec49108a6c98273a46a7c7" candidate_uuid: "34820d2299204f0c9600100869c0adfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" is_pre_election: true
I20260806 22:44:36.689162 666 raft_consensus.cc:2454] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34820d2299204f0c9600100869c0adfc in term 0.
I20260806 22:44:36.689437 462 leader_election.cc:302] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34820d2299204f0c9600100869c0adfc, ae02d9a743b94dedb7177540ab4ae8c4; no voters:
I20260806 22:44:36.689430 801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752bb645d0ec49108a6c98273a46a7c7" candidate_uuid: "34820d2299204f0c9600100869c0adfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b2a66da19db4467964b03d9b0c6723f" is_pre_election: true
I20260806 22:44:36.689564 801 raft_consensus.cc:2454] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34820d2299204f0c9600100869c0adfc in term 0.
I20260806 22:44:36.689790 889 raft_consensus.cc:2781] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:36.689870 889 raft_consensus.cc:482] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:36.689906 889 raft_consensus.cc:3037] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:36.690579 889 raft_consensus.cc:504] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.690770 889 leader_election.cc:288] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [CANDIDATE]: Term 1 election: Requested vote from peers 9b2a66da19db4467964b03d9b0c6723f (127.19.74.195:35975), ae02d9a743b94dedb7177540ab4ae8c4 (127.19.74.194:46603)
I20260806 22:44:36.690940 801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752bb645d0ec49108a6c98273a46a7c7" candidate_uuid: "34820d2299204f0c9600100869c0adfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b2a66da19db4467964b03d9b0c6723f"
I20260806 22:44:36.690943 666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752bb645d0ec49108a6c98273a46a7c7" candidate_uuid: "34820d2299204f0c9600100869c0adfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae02d9a743b94dedb7177540ab4ae8c4"
I20260806 22:44:36.691007 666 raft_consensus.cc:3037] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:36.691006 801 raft_consensus.cc:3037] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:36.691850 666 raft_consensus.cc:2454] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34820d2299204f0c9600100869c0adfc in term 1.
I20260806 22:44:36.691848 801 raft_consensus.cc:2454] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34820d2299204f0c9600100869c0adfc in term 1.
I20260806 22:44:36.692037 465 leader_election.cc:302] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34820d2299204f0c9600100869c0adfc, 9b2a66da19db4467964b03d9b0c6723f; no voters:
I20260806 22:44:36.692139 889 raft_consensus.cc:2781] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:36.692319 889 raft_consensus.cc:686] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [term 1 LEADER]: Becoming Leader. State: Replica: 34820d2299204f0c9600100869c0adfc, State: Running, Role: LEADER
I20260806 22:44:36.692417 889 consensus_queue.cc:237] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } }
I20260806 22:44:36.693307 375 catalog_manager.cc:5495] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc reported cstate change: term changed from 0 to 1, leader changed from <none> to 34820d2299204f0c9600100869c0adfc (127.19.74.193). New cstate: current_term: 1 leader_uuid: "34820d2299204f0c9600100869c0adfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34820d2299204f0c9600100869c0adfc" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 43583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 } health_report { overall_health: UNKNOWN } } }
I20260806 22:44:36.765317 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.762890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45578 (local address 127.19.74.193:43583)
0806 22:44:36.762952 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:36.762954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:36.762962 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:36.763004 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:36.763005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:36.763028 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:36.763108 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:36.763263 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.763756 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.764501 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:36.764720 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:36.764794 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:36.764981 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:36.765094 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:36.765185 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:36.765205 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:44:36.765225 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:44:36.767297 666 raft_consensus.cc:1261] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 1 FOLLOWER]: Refusing update from remote peer 34820d2299204f0c9600100869c0adfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:36.767360 801 raft_consensus.cc:1261] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 1 FOLLOWER]: Refusing update from remote peer 34820d2299204f0c9600100869c0adfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:36.767692 904 consensus_queue.cc:1035] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae02d9a743b94dedb7177540ab4ae8c4" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 46603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:36.767892 904 consensus_queue.cc:1035] T 752bb645d0ec49108a6c98273a46a7c7 P 34820d2299204f0c9600100869c0adfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b2a66da19db4467964b03d9b0c6723f" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:36.771767 907 mvcc.cc:205] Tried to move back new op lower bound from 7315686509636182016 to 7315686509332586496. Current Snapshot: MvccSnapshot[applied={T|T < 7315686509636182016}]
I20260806 22:44:36.773679 908 mvcc.cc:205] Tried to move back new op lower bound from 7315686509636182016 to 7315686509332586496. Current Snapshot: MvccSnapshot[applied={T|T < 7315686509636182016 or (T in {7315686509636182016})}]
I20260806 22:44:36.773861 910 mvcc.cc:205] Tried to move back new op lower bound from 7315686509636182016 to 7315686509332586496. Current Snapshot: MvccSnapshot[applied={T|T < 7315686509636182016}]
I20260806 22:44:36.776906 375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:36.776996 375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:36.777846 375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4512395368ec412bb697f8dc925ace88: Sending DeleteTablet for 3 replicas of tablet 752bb645d0ec49108a6c98273a46a7c7
I20260806 22:44:36.778276 510 tablet_service.cc:1514] Processing DeleteTablet for tablet 752bb645d0ec49108a6c98273a46a7c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:36 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:45574
I20260806 22:44:36.778474 781 tablet_service.cc:1514] Processing DeleteTablet for tablet 752bb645d0ec49108a6c98273a46a7c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:36 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:33970
I20260806 22:44:36.778884 646 tablet_service.cc:1514] Processing DeleteTablet for tablet 752bb645d0ec49108a6c98273a46a7c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:36 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:35674
I20260806 22:44:36.779379 919 tablet_replica.cc:330] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: stopping tablet replica
I20260806 22:44:36.779523 919 raft_consensus.cc:2229] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:36.779546 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 414
I20260806 22:44:36.779685 919 raft_consensus.cc:2258] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:36.779834 918 tablet_replica.cc:330] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: stopping tablet replica
I20260806 22:44:36.779918 918 raft_consensus.cc:2229] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:36.780005 918 raft_consensus.cc:2258] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:36.780646 918 ts_tablet_manager.cc:1905] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:36.782667 918 ts_tablet_manager.cc:1918] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:36.782786 918 log.cc:1196] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/752bb645d0ec49108a6c98273a46a7c7
I20260806 22:44:36.783048 919 ts_tablet_manager.cc:1905] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:36.783200 918 ts_tablet_manager.cc:1939] T 752bb645d0ec49108a6c98273a46a7c7 P 9b2a66da19db4467964b03d9b0c6723f: Deleting consensus metadata
I20260806 22:44:36.783666 363 catalog_manager.cc:4841] TS 9b2a66da19db4467964b03d9b0c6723f (127.19.74.195:35975): tablet 752bb645d0ec49108a6c98273a46a7c7 (table test-table [id=225525755f3947bcbf209c9eb469a529]) successfully deleted
I20260806 22:44:36.784680 919 ts_tablet_manager.cc:1918] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:36.784762 919 log.cc:1196] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/752bb645d0ec49108a6c98273a46a7c7
I20260806 22:44:36.785079 919 ts_tablet_manager.cc:1939] T 752bb645d0ec49108a6c98273a46a7c7 P ae02d9a743b94dedb7177540ab4ae8c4: Deleting consensus metadata
I20260806 22:44:36.785494 360 catalog_manager.cc:4841] TS ae02d9a743b94dedb7177540ab4ae8c4 (127.19.74.194:46603): tablet 752bb645d0ec49108a6c98273a46a7c7 (table test-table [id=225525755f3947bcbf209c9eb469a529]) successfully deleted
W20260806 22:44:36.790908 362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:43583 (error 108)
I20260806 22:44:36.791026 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 581
I20260806 22:44:36.796087 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:36.791895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:43583 (local address 127.0.0.1:45588)
0806 22:44:36.795991 (+ 4096us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:36.796041 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:43583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4070}
W20260806 22:44:36.796517 362 catalog_manager.cc:4584] TS 34820d2299204f0c9600100869c0adfc (127.19.74.193:43583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 752bb645d0ec49108a6c98273a46a7c7: Network error: Client connection negotiation failed: client connection to 127.19.74.193:43583: connect: Connection refused (error 111)
I20260806 22:44:36.800325 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 716
I20260806 22:44:36.806843 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 343
I20260806 22:44:36.817126 19755 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:44:37.161180 19755 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:44:37.161303 19755 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:44:37.505707 19755 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:44:37.505 UTC [31863] LOG: received smart shutdown request
2026-08-06 22:44:37.510 UTC [31863] LOG: background worker "logical replication launcher" (PID 31872) exited with exit code 1
2026-08-06 22:44:37.511 UTC [31867] LOG: shutting down
2026-08-06 22:44:37.661 UTC [31863] LOG: database system is shut down
2026-08-06T22:44:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47869 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:37 dist-test-slave-pgkr krb5kdc[930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:37 dist-test-slave-pgkr krb5kdc[930](info): set up 2 sockets
Aug 06 22:44:37 dist-test-slave-pgkr krb5kdc[930](info): commencing operation
krb5kdc: starting...
W20260806 22:44:38.813844 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:44:38 dist-test-slave-pgkr krb5kdc[930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:44:38.935288 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:46229
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:46229,127.19.74.253:42279,127.19.74.252:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:39.015592 943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:39.015805 943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:39.015842 943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:39.016924 943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:39.016983 943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:39.017005 943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:39.017025 943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:39.017051 943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:39.018569 943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:46229,127.19.74.253:42279,127.19.74.252:38695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:46229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:39.018851 943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:39.019064 943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:39.023687 950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:39.023813 943 server_base.cc:1034] running on GCE node
W20260806 22:44:39.023684 949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:39.023887 952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:39.024204 943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:39.024462 943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:39.025599 943 hybrid_clock.cc:648] HybridClock initialized: now 1786056279025582 us; error 39 us; skew 500 ppm
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:39.028020 943 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:44:39.028415 943 webserver.cc:460] Webserver started at http://127.19.74.254:43127/ using document root <none> and password file <none>
I20260806 22:44:39.028613 943 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:39.028653 943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:39.028743 943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:39.029739 943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "e9809e9ef47041a891e8048ac986024c"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "83c64c1947546fa5974520fa758ddfb9"
server_key_iv: "9ff3fee5b0f7f5a6445e728ea896e9c9"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.030036 943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "e9809e9ef47041a891e8048ac986024c"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "83c64c1947546fa5974520fa758ddfb9"
server_key_iv: "9ff3fee5b0f7f5a6445e728ea896e9c9"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.031399 943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:44:39.032155 959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:39.032333 943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:39.032462 943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "e9809e9ef47041a891e8048ac986024c"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "83c64c1947546fa5974520fa758ddfb9"
server_key_iv: "9ff3fee5b0f7f5a6445e728ea896e9c9"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.032546 943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:39.051046 943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:39.052930 943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:39.053100 943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:39.057324 943 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:46229
I20260806 22:44:39.057358 1011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:46229 every 8 connection(s)
I20260806 22:44:39.057744 943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:39.058656 1012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:39.059682 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 943
I20260806 22:44:39.059794 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:44:39.059980 19755 external_mini_cluster.cc:1428] Setting key a9ec66336d7e458fbd6f0ad05fa7f593
I20260806 22:44:39.060670 1012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.74.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.253@KRBTEST.COM" created.
I20260806 22:44:39.070861 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.061523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46229 (local address 127.0.0.1:50994)
0806 22:44:39.061702 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.061755 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.062325 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.062602 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.062605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.062907 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.063053 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.063057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.064072 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.064073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.064796 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.064798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.065298 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.065299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.065368 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.065873 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.065882 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.067586 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.069378 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.069386 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.069388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.069605 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.069822 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.069823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.069823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.069900 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.070232 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.070239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.070302 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.070410 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.070538 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260806 22:44:39.070871 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.061698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.19.74.254:46229)
0806 22:44:39.061994 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.061998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.062032 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:44:39.062486 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.062488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.062504 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.062597 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.063169 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.063972 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.064887 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.065209 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.065338 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.067677 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.067684 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.067685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.067691 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.069279 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.069680 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.069682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.069683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.069763 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.069952 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.069952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.069953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.070147 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.070208 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.070460 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.070647 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":262,"thread_start_us":63,"threads_started":1}
I20260806 22:44:39.071929 1012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:39.072264 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.072154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46216)
0806 22:44:39.072196 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.072247 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:44:39.072393 960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
I20260806 22:44:39.072865 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.072774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46224)
0806 22:44:39.072819 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.072845 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:44:39.072981 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:42279: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab.
Entry for principal kudu/127.19.74.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab.
I20260806 22:44:39.085384 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.253:42279
--webserver_interface=127.19.74.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab
--principal=kudu/127.19.74.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:46229,127.19.74.253:42279,127.19.74.252:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:39.122292 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } attempt: 1
I20260806 22:44:39.122849 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.122716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46240)
0806 22:44:39.122778 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.122821 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:44:39.123190 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.123112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46252)
0806 22:44:39.123151 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.123174 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:44:39.123286 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:42279: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
W20260806 22:44:39.167356 1019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:39.167546 1019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:39.167567 1019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:39.168627 1019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:39.168660 1019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:39.168674 1019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:39.168687 1019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:39.168699 1019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:39.170199 1019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:46229,127.19.74.253:42279,127.19.74.252:38695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab
--principal=kudu/127.19.74.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.253:42279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.74.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:39.170401 1019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:39.170565 1019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:39.175437 1025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:39.175449 1024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:39.175437 1027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:39.175674 1019 server_base.cc:1034] running on GCE node
I20260806 22:44:39.175906 1019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:39.176183 1019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:39.177318 1019 hybrid_clock.cc:648] HybridClock initialized: now 1786056279177308 us; error 28 us; skew 500 ppm
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:39.179877 1019 init.cc:377] Logged in from keytab as kudu/127.19.74.253@KRBTEST.COM (short username kudu)
I20260806 22:44:39.180243 1019 webserver.cc:460] Webserver started at http://127.19.74.253:33717/ using document root <none> and password file <none>
I20260806 22:44:39.180437 1019 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:39.180476 1019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:39.180560 1019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:39.181532 1019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/data/instance:
uuid: "a430b669cbc44b5799032f72dcf88589"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "4db94fba31aa90872188ba758527eacf"
server_key_iv: "6a5a9ff49df945505ad3e862dec8f202"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.181839 1019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/wal/instance:
uuid: "a430b669cbc44b5799032f72dcf88589"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "4db94fba31aa90872188ba758527eacf"
server_key_iv: "6a5a9ff49df945505ad3e862dec8f202"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.183135 1019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:39.183797 1034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:39.183959 1019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:39.184059 1019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/wal
uuid: "a430b669cbc44b5799032f72dcf88589"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "4db94fba31aa90872188ba758527eacf"
server_key_iv: "6a5a9ff49df945505ad3e862dec8f202"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.184135 1019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:39.191455 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } attempt: 2
I20260806 22:44:39.191833 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.191699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46268)
0806 22:44:39.191770 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.191804 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:44:39.192143 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.192051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46270)
0806 22:44:39.192093 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.192122 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:44:39.192241 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:42279: Network error: Client connection negotiation failed: client connection to 127.19.74.253:42279: connect: Connection refused (error 111)
I20260806 22:44:39.203765 1019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:39.205942 1019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:39.206102 1019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:39.210027 1019 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.253:42279
I20260806 22:44:39.210116 1088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.253:42279 every 8 connection(s)
I20260806 22:44:39.210433 1019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
I20260806 22:44:39.211333 1089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:39.212852 1089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:39.220330 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1019
I20260806 22:44:39.220432 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-1/wal/instance
I20260806 22:44:39.220669 19755 external_mini_cluster.cc:1428] Setting key 679365901b80baad0ba2905faf0dc0e5
I20260806 22:44:39.221271 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.213496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46229 (local address 127.0.0.1:51004)
0806 22:44:39.213674 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.213708 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.214122 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.214351 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.214354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.214578 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.214854 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.214858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.215483 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.215484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.216241 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.216242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.216606 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.216607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.216706 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.217099 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.217106 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.218368 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.219829 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.219835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.219836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.220021 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.220291 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.220292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.220293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.220421 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.220780 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.220786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.220917 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.221000 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.221122 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260806 22:44:39.221266 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.213544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.19.74.254:46229)
0806 22:44:39.213613 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.213616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.213782 (+ 166us) server_negotiation.cc:407] Connection header received
0806 22:44:39.214210 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.214211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.214223 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.214265 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.214927 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.215408 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.216309 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.216498 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.216561 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.218448 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.218450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.218450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.218456 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.219761 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.220093 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.220094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.220094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.220216 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.220502 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.220520 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.220521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.220672 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.220746 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.221045 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.221138 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:44:39.222048 1089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:44:39.229548 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.222308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46280)
0806 22:44:39.222353 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.222364 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.222409 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.222708 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.222709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.222867 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.222928 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.222929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.223738 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.223739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.224238 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.224239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.224852 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.224852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.224901 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.225314 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.225318 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.226533 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.228434 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.228434 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.228435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.228604 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.228843 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.228844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.228844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.228898 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.229188 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.229189 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.229263 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.229358 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.229450 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:44:39.229661 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.222372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.19.74.253:42279)
0806 22:44:39.222526 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.222532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.222553 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:44:39.222579 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.222581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.222595 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.222703 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.223002 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.223657 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.224317 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.224771 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.224840 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.226725 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.226734 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.226735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.226752 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.228333 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.228712 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.228715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.228716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.228778 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.228963 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.228964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.228965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.229105 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.229197 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.229389 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.229525 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.19.74.252@KRBTEST.COM; defaulting to no policy
I20260806 22:44:39.230139 1089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:39.230432 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.230328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35514)
0806 22:44:39.230372 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.230414 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:44:39.230568 1037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
I20260806 22:44:39.231074 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.230992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35524)
0806 22:44:39.231034 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.231058 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:44:39.231215 1089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:38695: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab.
Entry for principal kudu/127.19.74.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab.
I20260806 22:44:39.246718 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.252:38695
--webserver_interface=127.19.74.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab
--principal=kudu/127.19.74.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:46229,127.19.74.253:42279,127.19.74.252:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:39.280431 1089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } attempt: 1
I20260806 22:44:39.280933 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.280789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35530)
0806 22:44:39.280851 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.280904 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:44:39.281311 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.281186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35546)
0806 22:44:39.281240 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.281282 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:44:39.281466 1089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:38695: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
I20260806 22:44:39.283372 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } attempt: 3
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:44:39.290043 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.283712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.19.74.253:42279)
0806 22:44:39.283766 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.283769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.283779 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:44:39.283827 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.283828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.283837 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.283898 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.284251 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.284739 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.285440 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.285650 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.285722 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.287763 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.287764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.287764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.287770 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.289022 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.289411 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.289411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.289412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.289461 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.289647 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.289647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.289648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.289741 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.289787 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.289934 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.289990 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:44:39.290174 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.283639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46284)
0806 22:44:39.283691 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.283701 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.283765 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.283904 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.283905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.284082 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.284151 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.284153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.284824 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.284825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.285351 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.285352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.285719 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.285720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.285806 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.286299 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.286303 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.287673 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.289106 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.289107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.289108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.289343 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.289518 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.289519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.289519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.289586 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.289796 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.289797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.289845 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.289919 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.290011 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:44:39.290582 1012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:39.290927 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.290824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35556)
0806 22:44:39.290869 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.290903 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:44:39.291266 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.291181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35572)
0806 22:44:39.291222 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.291249 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:44:39.291354 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:38695: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
I20260806 22:44:39.322500 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } attempt: 1
I20260806 22:44:39.322952 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.322817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35584)
0806 22:44:39.322875 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.322916 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:44:39.323303 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.323201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35594)
0806 22:44:39.323247 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.323279 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:44:39.323406 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:38695: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
W20260806 22:44:39.332069 1097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:39.332257 1097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:39.332320 1097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:39.333465 1097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:39.333505 1097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:39.333519 1097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:39.333564 1097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:39.333582 1097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:39.335132 1097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:46229,127.19.74.253:42279,127.19.74.252:38695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab
--principal=kudu/127.19.74.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.252:38695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.74.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:39.335366 1097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:39.335584 1097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:39.340198 1102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:39.340209 1105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:39.340283 1097 server_base.cc:1034] running on GCE node
W20260806 22:44:39.340209 1103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:39.340678 1097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:39.340942 1097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:39.342082 1097 hybrid_clock.cc:648] HybridClock initialized: now 1786056279342059 us; error 37 us; skew 500 ppm
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:39.344856 1097 init.cc:377] Logged in from keytab as kudu/127.19.74.252@KRBTEST.COM (short username kudu)
I20260806 22:44:39.345273 1097 webserver.cc:460] Webserver started at http://127.19.74.252:34903/ using document root <none> and password file <none>
I20260806 22:44:39.345445 1097 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:39.345484 1097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:39.345569 1097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:39.346498 1097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/data/instance:
uuid: "c7bbbfdbed9e43619288943cc6250e0a"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "665564f63d048b63712d00efb9f58680"
server_key_iv: "bbe034e3b0507e7e77ffd81b1c583214"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.346881 1097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/wal/instance:
uuid: "c7bbbfdbed9e43619288943cc6250e0a"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "665564f63d048b63712d00efb9f58680"
server_key_iv: "bbe034e3b0507e7e77ffd81b1c583214"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.348104 1097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:39.348714 1112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:39.348868 1097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:39.348940 1097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/wal
uuid: "c7bbbfdbed9e43619288943cc6250e0a"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-pgkr"
server_key: "665564f63d048b63712d00efb9f58680"
server_key_iv: "bbe034e3b0507e7e77ffd81b1c583214"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.348989 1097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:39.349612 1089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } attempt: 2
I20260806 22:44:39.349964 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.349837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35596)
0806 22:44:39.349894 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.349931 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:44:39.350296 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.350214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35602)
0806 22:44:39.350251 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.350276 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 22:44:39.350383 1089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:38695: Network error: Client connection negotiation failed: client connection to 127.19.74.252:38695: connect: Connection refused (error 111)
I20260806 22:44:39.366132 1097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:39.368292 1097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:39.368448 1097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:39.372435 1097 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.252:38695
I20260806 22:44:39.372475 1165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.252:38695 every 8 connection(s)
I20260806 22:44:39.372895 1097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
I20260806 22:44:39.373893 1166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:39.375429 1166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:39.382292 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1097
I20260806 22:44:39.382359 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056127658350-19755-0/minicluster-data/master-2/wal/instance
I20260806 22:44:39.382555 19755 external_mini_cluster.cc:1428] Setting key 4c7f4edc172ea1495b072ac593dfacaa
I20260806 22:44:39.383527 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.376055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.19.74.254:46229)
0806 22:44:39.376124 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.376126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.376307 (+ 181us) server_negotiation.cc:407] Connection header received
0806 22:44:39.376728 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.376729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.376737 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.376794 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.377342 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.377804 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.378779 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.378971 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.379025 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.380992 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.380994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.380994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.381002 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.382133 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.382543 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.382544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.382544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.382628 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.382875 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.382877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.382877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.383027 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.383105 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.383400 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.383468 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:44:39.383800 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.376023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:46229 (local address 127.0.0.1:51014)
0806 22:44:39.376184 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.376224 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.376649 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.376867 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.376870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.377094 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.377274 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.377278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.377876 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.377877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.378704 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.378710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.379040 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.379040 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.379139 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.379600 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.379608 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.380898 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.382210 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.382217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.382218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.382476 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.382704 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.382704 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.382705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.382804 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.383098 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.383105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.383259 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.383375 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.383540 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260806 22:44:39.384379 1166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:44:39.391443 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.384618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:42279 (local address 127.0.0.1:46296)
0806 22:44:39.384729 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.384739 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.384788 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.384900 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.384900 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.385053 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.385102 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.385104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.385942 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.385943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.386681 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.386683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.387041 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.387042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.387103 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.387476 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.387480 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.388660 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.390367 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.390368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.390369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.390698 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.390884 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.390884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.390885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.390943 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.391143 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.391144 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.391186 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.391267 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.391349 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 22:44:39.391456 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.384682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.19.74.253:42279)
0806 22:44:39.384740 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.384742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.384774 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:44:39.384843 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.384844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.384851 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.384895 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.385169 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.385872 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.386759 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.386975 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.387044 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.388744 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.388745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.388746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.388752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.390253 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.390765 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.390765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.390766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.390818 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.390999 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.390999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.390999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.391087 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.391142 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.391282 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.391337 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:44:39.391783 1166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:39.392793 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.382923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.19.74.252:38695)
0806 22:44:39.383085 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.383101 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.383122 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:44:39.383174 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.383176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.383187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.383244 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.383949 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.384936 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.385939 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.386367 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.386447 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.389566 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.389573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.389573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.389579 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.391307 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.391678 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.391682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.391682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.391736 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.392175 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.392177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.392177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.392365 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.392490 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.392582 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.392675 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:39.398532 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } attempt: 2
I20260806 22:44:39.399102 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.392014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35616)
0806 22:44:39.392052 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.392060 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.392089 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.392319 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.392320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.392419 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.392444 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.392445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.393159 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.393160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.393731 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.393732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.394094 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.394095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.394139 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.394539 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.394543 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.395834 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.398093 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.398094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.398094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.398250 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.398453 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.398454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.398454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.398506 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.398793 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.398794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.398830 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.398911 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.399001 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:44:39.399180 1171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.392052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.19.74.252:38695)
0806 22:44:39.392187 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.392188 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.392195 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:44:39.392212 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.392212 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.392222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.392303 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.392520 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.393075 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.393787 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.394017 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.394093 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.395901 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.395902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.395902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.395909 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.397995 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.398332 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.398333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.398333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.398387 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.398567 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.398568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.398568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.398725 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.398800 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.398935 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.399049 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260806 22:44:39.400939 1166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a: Bootstrap starting.
I20260806 22:44:39.401100 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.393549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.19.74.254:46229)
0806 22:44:39.393583 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.393585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.393596 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:44:39.393617 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.393618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.393626 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.393691 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.394093 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.394797 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.395743 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.395960 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.396035 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.397982 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.397983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.397983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.397990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.399927 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.400307 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.400308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.400308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.400363 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.400562 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.400563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.400563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.400725 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.400900 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.400941 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.401012 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":18}
I20260806 22:44:39.401718 1166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:39.402022 1166 log.cc:824] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:39.402817 1166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a: No bootstrap required, opened a new log
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:44:39.405010 1166 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.405161 1166 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:39.405217 1166 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bbbfdbed9e43619288943cc6250e0a, State: Initialized, Role: FOLLOWER
I20260806 22:44:39.405364 1166 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.405633 1173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.405692 1166 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:39.405799 1173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:44:39.406778 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.398769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35626)
0806 22:44:39.399077 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.399088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.399140 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.399386 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.399387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.399662 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.399702 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.399705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.400621 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.400622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.401370 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.401372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.401852 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.401853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.401911 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.402341 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.402346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.403743 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.405584 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.405585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.405586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.405786 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.406045 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.406046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.406047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.406120 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.406440 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.406441 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.406488 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.406565 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.406722 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291}
I20260806 22:44:39.407070 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.399242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.19.74.252:38695)
0806 22:44:39.399278 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.399279 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.399287 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:39.399305 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.399305 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.399314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.399370 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.399789 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.400510 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.401458 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.401765 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.402694 (+ 929us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.403842 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.403844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.403844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.403851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.405485 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.405875 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.405875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.405876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.405928 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.406195 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.406196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.406197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.406369 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.406446 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.406888 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.406978 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:44:39.408278 1185 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:39.408329 1185 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:39.409020 1012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: Bootstrap starting.
I20260806 22:44:39.409622 1012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:39.409911 1012 log.cc:824] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:39.409972 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.401675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.19.74.253:42279)
0806 22:44:39.401710 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.401711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.401720 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:44:39.401739 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.401739 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.401747 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.401803 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.402455 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.403101 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.403806 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.404009 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.404080 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.406578 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.406579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.406579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.406585 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.408460 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.408844 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.408845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.408845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.408899 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.409081 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.409082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.409082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.409216 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.409404 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.409810 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.409883 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:44:39.410881 1012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: No bootstrap required, opened a new log
I20260806 22:44:39.408675 1097 master_runner.cc:186] Error getting master registration for 127.19.74.254:46229: OK, instance_id { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" instance_seqno: 1786056279056100 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:44:39.411083 1097 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:44:39.412012 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:39.413007 1012 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.413152 1012 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:39.413198 1012 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9809e9ef47041a891e8048ac986024c, State: Initialized, Role: FOLLOWER
I20260806 22:44:39.413321 1012 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.413636 1012 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:39.415057 1192 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.415452 1192 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:44:39.415653 1205 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:39.415705 1205 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:39.416314 943 master_runner.cc:186] Error getting master registration for 127.19.74.253:42279: OK, instance_id { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" instance_seqno: 1786056279209015 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:44:39.419184 943 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:44:39.427129 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.413556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.19.74.253:42279)
0806 22:44:39.413595 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.413597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.418734 (+ 5137us) server_negotiation.cc:407] Connection header received
0806 22:44:39.418840 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.418841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.418853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.418884 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.419624 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.420318 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.421339 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.421639 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.421709 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.424091 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.424093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.424093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.424100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.425788 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.426215 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.426217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.426217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.426277 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.426491 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.426492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.426493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.426661 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.426747 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.426961 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.427038 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:44:39.430858 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.416108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.19.74.254:46229)
0806 22:44:39.416143 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.416145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.421446 (+ 5301us) server_negotiation.cc:407] Connection header received
0806 22:44:39.421534 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.421535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.421545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.421575 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.422279 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.422968 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.424887 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.425172 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.425210 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.427807 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.427808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.427809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.427815 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.429526 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.429939 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.429940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.429940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.430001 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.430216 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.430217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.430217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.430336 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.430413 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.430683 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.430760 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:44:39.431840 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.415776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.19.74.252:38695)
0806 22:44:39.415932 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.415934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.422353 (+ 6419us) server_negotiation.cc:407] Connection header received
0806 22:44:39.422441 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.422442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.422452 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.422479 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.424060 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.424790 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.426436 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.426762 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.426805 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.428950 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.428952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.428952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.428959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.430562 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.431000 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.431001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.431001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.431052 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.431248 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.431249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.431249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.431363 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.431443 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.431666 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.431750 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260806 22:44:39.441505 1089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } attempt: 3
Aug 06 22:44:39 dist-test-slave-pgkr krb5kdc[930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:44:39.448873 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.441748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:38695 (local address 127.0.0.1:35654)
0806 22:44:39.441790 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:39.441800 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:39.441833 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:39.441959 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:39.441960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:39.442109 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:39.442146 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.442147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.442985 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.442986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.443730 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:39.443732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.444148 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.444149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:39.444203 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.444571 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:39.444575 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:39.446003 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:39.447785 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.447786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.447787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.448019 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.448261 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:39.448261 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:39.448262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:39.448316 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:39.448572 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:39.448573 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:39.448625 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:44:39.448686 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:44:39.448772 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:44:39.449206 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.441792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.19.74.252:38695)
0806 22:44:39.441826 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.441828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.441836 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:39.441880 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.441881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.441889 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.441948 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.442218 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.442904 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.443808 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.444073 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.444140 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.446101 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.446103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.446103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.446109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.447704 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.448136 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.448137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.448138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.448193 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.448379 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.448380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.448380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.448493 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.448548 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.449049 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.449120 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:44:39.450695 1089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589: Bootstrap starting.
I20260806 22:44:39.451478 1089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:39.451845 1089 log.cc:824] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:39.452775 1089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589: No bootstrap required, opened a new log
I20260806 22:44:39.454752 1089 raft_consensus.cc:348] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.454886 1089 raft_consensus.cc:374] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:39.454919 1089 raft_consensus.cc:729] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a430b669cbc44b5799032f72dcf88589, State: Initialized, Role: FOLLOWER
I20260806 22:44:39.455052 1089 consensus_queue.cc:260] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.455304 1210 sys_catalog.cc:455] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.455353 1089 sys_catalog.cc:564] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 22:44:39.456688 1040 tablet.cc:2369] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:44:39.457696 1221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:39.457804 1221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:39.455436 1210 sys_catalog.cc:458] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:39.459757 1019 master_runner.cc:263] No leader master found from master a430b669cbc44b5799032f72dcf88589
I20260806 22:44:39.459811 1019 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 22:44:39.553872 965 tablet.cc:2369] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:44:39.553962 1210 raft_consensus.cc:482] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:39.554042 1210 raft_consensus.cc:504] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.554473 1140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a430b669cbc44b5799032f72dcf88589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bbbfdbed9e43619288943cc6250e0a" is_pre_election: true
I20260806 22:44:39.554487 986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a430b669cbc44b5799032f72dcf88589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9809e9ef47041a891e8048ac986024c" is_pre_election: true
I20260806 22:44:39.554642 986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a430b669cbc44b5799032f72dcf88589 in term 0.
I20260806 22:44:39.554664 1140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a430b669cbc44b5799032f72dcf88589 in term 0.
I20260806 22:44:39.554837 1038 leader_election.cc:302] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a430b669cbc44b5799032f72dcf88589, e9809e9ef47041a891e8048ac986024c; no voters:
I20260806 22:44:39.554965 1210 leader_election.cc:288] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9809e9ef47041a891e8048ac986024c (127.19.74.254:46229), c7bbbfdbed9e43619288943cc6250e0a (127.19.74.252:38695)
I20260806 22:44:39.555027 1210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:39.555068 1210 raft_consensus.cc:482] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:39.555094 1210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:39.555784 1210 raft_consensus.cc:504] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.556042 986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a430b669cbc44b5799032f72dcf88589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9809e9ef47041a891e8048ac986024c"
I20260806 22:44:39.556097 1140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a430b669cbc44b5799032f72dcf88589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bbbfdbed9e43619288943cc6250e0a"
I20260806 22:44:39.556133 986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:39.556159 1140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:39.556893 1140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a430b669cbc44b5799032f72dcf88589 in term 1.
I20260806 22:44:39.556893 986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a430b669cbc44b5799032f72dcf88589 in term 1.
I20260806 22:44:39.557036 1038 leader_election.cc:302] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a430b669cbc44b5799032f72dcf88589, e9809e9ef47041a891e8048ac986024c; no voters:
I20260806 22:44:39.557152 1223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:39.557160 1210 leader_election.cc:288] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [CANDIDATE]: Term 1 election: Requested vote from peers e9809e9ef47041a891e8048ac986024c (127.19.74.254:46229), c7bbbfdbed9e43619288943cc6250e0a (127.19.74.252:38695)
I20260806 22:44:39.557456 1223 raft_consensus.cc:686] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [term 1 LEADER]: Becoming Leader. State: Replica: a430b669cbc44b5799032f72dcf88589, State: Running, Role: LEADER
I20260806 22:44:39.557550 1223 consensus_queue.cc:237] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } }
I20260806 22:44:39.557948 1210 sys_catalog.cc:455] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a430b669cbc44b5799032f72dcf88589. Latest consensus state: current_term: 1 leader_uuid: "a430b669cbc44b5799032f72dcf88589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.558022 1210 sys_catalog.cc:458] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:39.558235 1227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:39.558451 1227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:39.559738 1140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [term 1 FOLLOWER]: Refusing update from remote peer a430b669cbc44b5799032f72dcf88589: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:39.559732 986 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [term 1 FOLLOWER]: Refusing update from remote peer a430b669cbc44b5799032f72dcf88589: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:39.560011 1223 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:39.560093 1210 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:39.561215 1192 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a430b669cbc44b5799032f72dcf88589. Latest consensus state: current_term: 1 leader_uuid: "a430b669cbc44b5799032f72dcf88589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.561221 1173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a430b669cbc44b5799032f72dcf88589. Latest consensus state: current_term: 1 leader_uuid: "a430b669cbc44b5799032f72dcf88589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.561297 1173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:39.561297 1192 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:39.562180 1223 sys_catalog.cc:455] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a430b669cbc44b5799032f72dcf88589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.562256 1223 sys_catalog.cc:458] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:39.562297 1223 sys_catalog.cc:455] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a430b669cbc44b5799032f72dcf88589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.562332 1223 sys_catalog.cc:458] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:39.562426 1192 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a430b669cbc44b5799032f72dcf88589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.562508 1192 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:39.562573 1228 mvcc.cc:205] Tried to move back new op lower bound from 7315686521074696192 to 7315686521068384256. Current Snapshot: MvccSnapshot[applied={T|T < 7315686521074696192}]
I20260806 22:44:39.562829 1173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a430b669cbc44b5799032f72dcf88589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9809e9ef47041a891e8048ac986024c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46229 } } peers { permanent_uuid: "a430b669cbc44b5799032f72dcf88589" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 42279 } } peers { permanent_uuid: "c7bbbfdbed9e43619288943cc6250e0a" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 38695 } } }
I20260806 22:44:39.562896 1173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:39.563086 1229 mvcc.cc:205] Tried to move back new op lower bound from 7315686521074696192 to 7315686521068384256. Current Snapshot: MvccSnapshot[applied={T|T < 7315686521074696192}]
I20260806 22:44:39.563557 1227 catalog_manager.cc:1347] Generated new cluster ID: 9b8542198da94f9386133b5debe8cd47
I20260806 22:44:39.563625 1227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:39.563731 1230 mvcc.cc:205] Tried to move back new op lower bound from 7315686521074696192 to 7315686521068384256. Current Snapshot: MvccSnapshot[applied={T|T < 7315686521074696192}]
I20260806 22:44:39.578023 1227 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:39.578802 1227 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:39.585973 1227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a430b669cbc44b5799032f72dcf88589: Generated new TSK 0
I20260806 22:44:39.586153 1227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:40.408802 1185 catalog_manager.cc:1259] Loaded cluster ID: 9b8542198da94f9386133b5debe8cd47
I20260806 22:44:40.408903 1185 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a: loading cluster ID for follower catalog manager: success
I20260806 22:44:40.409735 1185 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a: acquiring CA information for follower catalog manager: success
I20260806 22:44:40.410030 1185 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c7bbbfdbed9e43619288943cc6250e0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:44:40.416061 1205 catalog_manager.cc:1259] Loaded cluster ID: 9b8542198da94f9386133b5debe8cd47
I20260806 22:44:40.416127 1205 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: loading cluster ID for follower catalog manager: success
I20260806 22:44:40.416850 1205 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: acquiring CA information for follower catalog manager: success
I20260806 22:44:40.417122 1205 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e9809e9ef47041a891e8048ac986024c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:44:40.632125 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 943
I20260806 22:44:40.639712 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1019
I20260806 22:44:40.645565 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1097
2026-08-06T22:44:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2869 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:40 dist-test-slave-pgkr krb5kdc[1241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:40 dist-test-slave-pgkr krb5kdc[1241](info): set up 2 sockets
Aug 06 22:44:40 dist-test-slave-pgkr krb5kdc[1241](info): commencing operation
krb5kdc: starting...
W20260806 22:44:41.689144 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:44:41 dist-test-slave-pgkr krb5kdc[1241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:44:41.808740 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34111
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:37123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:34111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:41.890323 1254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:41.890509 1254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:41.890529 1254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:41.891706 1254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:41.891748 1254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:41.891763 1254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:41.891776 1254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:41.891788 1254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:41.893193 1254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:37123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:34111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:41.893383 1254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:41.893563 1254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:41.898233 1262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:41.898352 1254 server_base.cc:1034] running on GCE node
W20260806 22:44:41.898247 1260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:41.898458 1259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:41.898746 1254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:41.899035 1254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:41.900198 1254 hybrid_clock.cc:648] HybridClock initialized: now 1786056281900192 us; error 34 us; skew 500 ppm
Aug 06 22:44:41 dist-test-slave-pgkr krb5kdc[1241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056281, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:41.902484 1254 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:44:41.902904 1254 webserver.cc:460] Webserver started at http://127.19.74.254:37803/ using document root <none> and password file <none>
I20260806 22:44:41.903121 1254 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:41.903184 1254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:41.903326 1254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:41.904299 1254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "b1582871719345a28d26ab7031322294"
format_stamp: "Formatted at 2026-08-06 22:44:41 on dist-test-slave-pgkr"
server_key: "072b19a76395f586848eda1cde6f8e2b"
server_key_iv: "2d98751994566e6d388fe9a7d4be9856"
server_key_version: "encryptionkey@0"
I20260806 22:44:41.904663 1254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "b1582871719345a28d26ab7031322294"
format_stamp: "Formatted at 2026-08-06 22:44:41 on dist-test-slave-pgkr"
server_key: "072b19a76395f586848eda1cde6f8e2b"
server_key_iv: "2d98751994566e6d388fe9a7d4be9856"
server_key_version: "encryptionkey@0"
I20260806 22:44:41.905941 1254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:41.906605 1269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:41.906810 1254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:41.906941 1254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "b1582871719345a28d26ab7031322294"
format_stamp: "Formatted at 2026-08-06 22:44:41 on dist-test-slave-pgkr"
server_key: "072b19a76395f586848eda1cde6f8e2b"
server_key_iv: "2d98751994566e6d388fe9a7d4be9856"
server_key_version: "encryptionkey@0"
I20260806 22:44:41.907008 1254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:41.935853 1254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:41.937724 1254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:41.937887 1254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:41.941766 1254 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:34111
I20260806 22:44:41.941823 1322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:34111 every 8 connection(s)
I20260806 22:44:41.942171 1254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:41.943140 1323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:41.943259 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1254
I20260806 22:44:41.943342 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:44:41.943573 19755 external_mini_cluster.cc:1428] Setting key 2d01338d49bfdfacaea4f036f445a401
I20260806 22:44:41.946401 1323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294: Bootstrap starting.
I20260806 22:44:41.947223 1323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:41.947511 1323 log.cc:824] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:41.948244 1323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294: No bootstrap required, opened a new log
Aug 06 22:44:41 dist-test-slave-pgkr krb5kdc[1241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:41.950491 1323 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1582871719345a28d26ab7031322294" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34111 } }
I20260806 22:44:41.950692 1323 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:41.950744 1323 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1582871719345a28d26ab7031322294, State: Initialized, Role: FOLLOWER
I20260806 22:44:41.950891 1323 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1582871719345a28d26ab7031322294" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34111 } }
I20260806 22:44:41.951010 1323 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:41.951056 1323 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:41.951103 1323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:41.951778 1323 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1582871719345a28d26ab7031322294" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34111 } }
I20260806 22:44:41.951895 1323 leader_election.cc:302] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1582871719345a28d26ab7031322294; no voters:
I20260806 22:44:41.952077 1323 leader_election.cc:288] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:41.952126 1329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:41.952303 1329 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [term 1 LEADER]: Becoming Leader. State: Replica: b1582871719345a28d26ab7031322294, State: Running, Role: LEADER
I20260806 22:44:41.952365 1323 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:41.953023 1329 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1582871719345a28d26ab7031322294" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34111 } }
I20260806 22:44:41.953652 1331 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1582871719345a28d26ab7031322294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1582871719345a28d26ab7031322294" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34111 } } }
I20260806 22:44:41.953665 1330 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1582871719345a28d26ab7031322294. Latest consensus state: current_term: 1 leader_uuid: "b1582871719345a28d26ab7031322294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1582871719345a28d26ab7031322294" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34111 } } }
I20260806 22:44:41.954010 1330 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:41.953881 1331 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294 [sys.catalog]: This master's current role is: LEADER
W20260806 22:44:41.954324 1343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:41.954429 1343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:41.954559 1345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:41.954815 1345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:41.955008 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:41.944076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.19.74.254:34111)
0806 22:44:41.944235 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:41.944243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:41.944268 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:44:41.944725 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:41.944731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:41.944772 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:41.944874 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:41.945613 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:41.946537 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:41.947672 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:41.948202 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:41.948314 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:41.950881 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:41.950884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:41.950885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:41.950896 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:41.952799 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:41.953217 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:41.953220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:41.953221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:41.953328 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:41.953559 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:41.953560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:41.953560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:41.953906 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:41.954052 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:41.954554 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 22:44:41.954727 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260806 22:44:41.956007 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:41.956451 1345 catalog_manager.cc:1347] Generated new cluster ID: a15f955be51c44e2b4637517a80dd3e0
I20260806 22:44:41.956498 1345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:41.961529 1345 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:41.962023 1345 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:41.966204 1345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1582871719345a28d26ab7031322294: Generated new TSK 0
I20260806 22:44:41.966360 1345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:42.009960 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1254
2026-08-06T22:44:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1364 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:42 dist-test-slave-pgkr krb5kdc[1354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:42 dist-test-slave-pgkr krb5kdc[1354](info): set up 2 sockets
Aug 06 22:44:42 dist-test-slave-pgkr krb5kdc[1354](info): commencing operation
krb5kdc: starting...
W20260806 22:44:43.044488 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:44:43.163542 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42801
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:42801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:43.250571 1367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:43.250909 1367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:43.250952 1367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:43.252771 1367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:43.252841 1367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:43.252918 1367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:43.252948 1367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:43.252965 1367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:43.254400 1367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:42801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:43.254705 1367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:43.254944 1367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:43.259821 1372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:43.259845 1375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:43.259845 1373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:43.260106 1367 server_base.cc:1034] running on GCE node
I20260806 22:44:43.260305 1367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:43.260583 1367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:43.261729 1367 hybrid_clock.cc:648] HybridClock initialized: now 1786056283261715 us; error 28 us; skew 500 ppm
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:43.269083 1367 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:44:43.269548 1367 webserver.cc:460] Webserver started at http://127.19.74.254:33199/ using document root <none> and password file <none>
I20260806 22:44:43.269791 1367 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:43.269834 1367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:43.269976 1367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:43.271109 1367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "527a9efe72ec4d228b5bdb08b0270bc6"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "dd2585eb7cd7dc750c4dc69272331351"
server_key_iv: "d3e9135de52ac62d6566550747309fca"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.271481 1367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "527a9efe72ec4d228b5bdb08b0270bc6"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "dd2585eb7cd7dc750c4dc69272331351"
server_key_iv: "d3e9135de52ac62d6566550747309fca"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.272815 1367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:43.273527 1382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.273716 1367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:43.273819 1367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "527a9efe72ec4d228b5bdb08b0270bc6"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "dd2585eb7cd7dc750c4dc69272331351"
server_key_iv: "d3e9135de52ac62d6566550747309fca"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.273901 1367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:43.288169 1367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:43.290026 1367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:43.290179 1367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:43.294356 1367 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:42801
I20260806 22:44:43.294391 1435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:42801 every 8 connection(s)
I20260806 22:44:43.294811 1367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:43.295782 1436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.298168 1436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Bootstrap starting.
I20260806 22:44:43.298416 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1367
I20260806 22:44:43.298795 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:44:43.298916 1436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.299055 19755 external_mini_cluster.cc:1428] Setting key f70fafc156fdf65f2667ecb85819397b
I20260806 22:44:43.299299 1436 log.cc:824] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:43.300179 1436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: No bootstrap required, opened a new log
I20260806 22:44:43.302745 1436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:43.302927 1436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.302960 1436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527a9efe72ec4d228b5bdb08b0270bc6, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.303094 1436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:43.303300 1436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:43.303361 1436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:43.303422 1436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.304071 1436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:43.304179 1436 leader_election.cc:302] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 527a9efe72ec4d228b5bdb08b0270bc6; no voters:
I20260806 22:44:43.304342 1436 leader_election.cc:288] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:43.304401 1441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:43.304602 1441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 1 LEADER]: Becoming Leader. State: Replica: 527a9efe72ec4d228b5bdb08b0270bc6, State: Running, Role: LEADER
I20260806 22:44:43.304734 1441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:43.304617 1436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:43.305116 1441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 527a9efe72ec4d228b5bdb08b0270bc6. Latest consensus state: current_term: 1 leader_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } } }
I20260806 22:44:43.305362 1441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:43.305158 1442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } } }
I20260806 22:44:43.305629 1442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:43.305992 1446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:43.306469 1446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:43.311062 1446 catalog_manager.cc:1347] Generated new cluster ID: 64864ccfd1d74071843f3721a9086505
I20260806 22:44:43.311080 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.299649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.19.74.254:42801)
0806 22:44:43.299842 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.299851 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.299884 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:44:43.300518 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.300528 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.300592 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.300744 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.301445 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.302987 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.303867 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.304540 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.304701 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.307716 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.307722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.307722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.307734 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.309427 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.309859 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.309861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.309861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.309993 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.310227 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.310227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.310228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.310437 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.310550 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.310733 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.310865 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":107,"threads_started":1}
I20260806 22:44:43.311129 1446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:43.329924 1446 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:43.330662 1446 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:43.335083 1446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Generated new TSK 0
I20260806 22:44:43.335283 1446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:44:43.365065 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42801
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:43.459290 1463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:43.459470 1463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:43.459493 1463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:43.460608 1463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:43.460646 1463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:43.460692 1463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:44:43.462095 1463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:43.462342 1463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:43.462568 1463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:43.467394 1471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:43.467386 1469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:43.467391 1468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:43.467429 1463 server_base.cc:1034] running on GCE node
I20260806 22:44:43.467847 1463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:43.468082 1463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:43.469233 1463 hybrid_clock.cc:648] HybridClock initialized: now 1786056283469220 us; error 28 us; skew 500 ppm
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:43.471827 1463 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:44:43.472196 1463 webserver.cc:460] Webserver started at http://127.19.74.193:33981/ using document root <none> and password file <none>
I20260806 22:44:43.472383 1463 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:43.472422 1463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:43.472504 1463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:43.473415 1463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "7304c1c03da94150b6303fd361f3be6d"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "bda80debb0b1e93fff101bc849ba78cb"
server_key_iv: "07f82c88317dc808f2f3a753c1b73b3d"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.473697 1463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "7304c1c03da94150b6303fd361f3be6d"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "bda80debb0b1e93fff101bc849ba78cb"
server_key_iv: "07f82c88317dc808f2f3a753c1b73b3d"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.474916 1463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:44:43.475564 1478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.475732 1463 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:43.475816 1463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "7304c1c03da94150b6303fd361f3be6d"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "bda80debb0b1e93fff101bc849ba78cb"
server_key_iv: "07f82c88317dc808f2f3a753c1b73b3d"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.475883 1463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:43.489141 1463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:43.490823 1463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:43.490959 1463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:43.491143 1463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:43.491436 1463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:43.491484 1463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.491523 1463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:43.491549 1463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.497186 1463 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:46523
I20260806 22:44:43.497275 1595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:46523 every 8 connection(s)
I20260806 22:44:43.497550 1463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:43.499580 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1463
I20260806 22:44:43.499689 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:43.499873 19755 external_mini_cluster.cc:1428] Setting key 978227c19a9bc315d53a31e2639052e1
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:43.507993 1599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.498212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:50791)
0806 22:44:43.498392 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.498429 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.498915 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.499119 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.499122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.499860 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:43.500131 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.500138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.500959 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.500960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.502232 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.502235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.502654 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.502655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.502782 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.503260 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:43.503268 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:43.504608 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:43.506720 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.506727 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.506730 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.506953 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.507168 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.507169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.507169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.507259 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.507525 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:43.507529 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:43.507655 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.507733 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.507844 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260806 22:44:43.507987 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.498254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:50791 (local address 127.19.74.254:42801)
0806 22:44:43.498360 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.498362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.498510 (+ 148us) server_negotiation.cc:407] Connection header received
0806 22:44:43.498999 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.499000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.499010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.499059 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.500243 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.500869 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.502315 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.502528 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.502603 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.504705 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.504707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.504707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.504717 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.506557 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.507024 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.507026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.507026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.507098 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.507316 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.507317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.507318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.507459 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.507536 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.507780 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.507880 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:44:43.508553 1597 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42801
I20260806 22:44:43.508646 1597 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:43.508836 1597 heartbeater.cc:511] Master 127.19.74.254:42801 requested a full tablet report, sending...
I20260806 22:44:43.509352 1399 ts_manager.cc:194] Registered new tserver with Master: 7304c1c03da94150b6303fd361f3be6d (127.19.74.193:46523)
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
I20260806 22:44:43.509897 1399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:50791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:44:43.525313 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42801
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:43.607553 1602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:43.607764 1602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:43.607789 1602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:43.608892 1602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:43.608930 1602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:43.608956 1602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:44:43.610399 1602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:43.610584 1602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:43.610788 1602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:44:43.615701 1602 server_base.cc:1034] running on GCE node
W20260806 22:44:43.615684 1611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:43.615667 1609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:43.615677 1608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:43.616082 1602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:43.616331 1602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:43.617489 1602 hybrid_clock.cc:648] HybridClock initialized: now 1786056283617478 us; error 28 us; skew 500 ppm
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:43.620023 1602 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:44:43.620433 1602 webserver.cc:460] Webserver started at http://127.19.74.194:33945/ using document root <none> and password file <none>
I20260806 22:44:43.620652 1602 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:43.620716 1602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:43.620857 1602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:43.621853 1602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "3b0dde0622c74831a930dd9098982878"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "131b219050e24aacdf4646dbe3f6d6e9"
server_key_iv: "bd3ab8d33f919bb0c78b533fad4cbaa7"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.622205 1602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "3b0dde0622c74831a930dd9098982878"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "131b219050e24aacdf4646dbe3f6d6e9"
server_key_iv: "bd3ab8d33f919bb0c78b533fad4cbaa7"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.623569 1602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:43.624284 1618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.624450 1602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:43.624552 1602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "3b0dde0622c74831a930dd9098982878"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "131b219050e24aacdf4646dbe3f6d6e9"
server_key_iv: "bd3ab8d33f919bb0c78b533fad4cbaa7"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.624621 1602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:43.641294 1602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:43.643517 1602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:43.643668 1602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:43.643877 1602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:43.644198 1602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:43.644248 1602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.644280 1602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:43.644305 1602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.649880 1602 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:43867
I20260806 22:44:43.649943 1733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:43867 every 8 connection(s)
I20260806 22:44:43.650257 1602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:43.658545 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.650934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:44221 (local address 127.19.74.254:42801)
0806 22:44:43.650995 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.650997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.651160 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:44:43.651607 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.651608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.651618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.651661 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.652528 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.653000 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.653909 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.654077 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.654126 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.656104 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.656105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.656105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.656117 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.657455 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.657820 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.657821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.657822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.657869 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.658054 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.658054 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.658055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.658155 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.658203 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.658423 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.658484 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:44:43.658735 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.650885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:44221)
0806 22:44:43.651036 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.651071 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.651515 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.651711 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.651714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.652264 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:43.652458 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.652462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.653073 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.653074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.653842 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.653843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.654150 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.654151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.654241 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.654710 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:43.654719 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:43.656014 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:43.657540 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.657546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.657548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.657748 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.657920 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.657920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.657921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.658007 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.658203 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:43.658207 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:43.658322 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.658399 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.658510 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260806 22:44:43.659174 1734 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42801
I20260806 22:44:43.659260 1734 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:43.659391 1734 heartbeater.cc:511] Master 127.19.74.254:42801 requested a full tablet report, sending...
I20260806 22:44:43.659754 1399 ts_manager.cc:194] Registered new tserver with Master: 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867)
I20260806 22:44:43.660032 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1602
I20260806 22:44:43.660130 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:43.660131 1399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:44221
I20260806 22:44:43.660319 19755 external_mini_cluster.cc:1428] Setting key 39310bba7ac86086f56c6cf1c9dcfcc3
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:44:43.684805 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42801
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:43.767544 1739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:43.767787 1739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:43.767827 1739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:43.768958 1739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:43.769026 1739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:43.769068 1739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:44:43.770634 1739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:43.770903 1739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:43.771136 1739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:43.776249 1745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:43.776291 1747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:43.776382 1744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:43.776703 1739 server_base.cc:1034] running on GCE node
I20260806 22:44:43.776922 1739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:43.777194 1739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:43.778381 1739 hybrid_clock.cc:648] HybridClock initialized: now 1786056283778352 us; error 41 us; skew 500 ppm
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:43.781154 1739 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:44:43.781664 1739 webserver.cc:460] Webserver started at http://127.19.74.195:38703/ using document root <none> and password file <none>
I20260806 22:44:43.781883 1739 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:43.781934 1739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:43.782050 1739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:43.783231 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "daedd8e176d64bba9b5dc8e3d7392f1b"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "188bf9e01d0693112033bdab45d77073"
server_key_iv: "2ad870bb132507ca656844e3dba67109"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.783586 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "daedd8e176d64bba9b5dc8e3d7392f1b"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "188bf9e01d0693112033bdab45d77073"
server_key_iv: "2ad870bb132507ca656844e3dba67109"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.785060 1739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:43.785830 1755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.786015 1739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:43.786099 1739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "daedd8e176d64bba9b5dc8e3d7392f1b"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "188bf9e01d0693112033bdab45d77073"
server_key_iv: "2ad870bb132507ca656844e3dba67109"
server_key_version: "encryptionkey@0"
I20260806 22:44:43.786160 1739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:43.796386 1739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:43.798458 1739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:43.798584 1739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:43.798830 1739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:43.799085 1739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:43.799115 1739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.799135 1739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:43.799149 1739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.804855 1739 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:43235
I20260806 22:44:43.804906 1872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:43235 every 8 connection(s)
I20260806 22:44:43.805269 1739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:43.810043 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1739
I20260806 22:44:43.810175 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:43.810355 19755 external_mini_cluster.cc:1428] Setting key 32a1d3ca372cb93b0a1997816ffd5a59
Aug 06 22:44:43 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:43.814458 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.806010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:43693 (local address 127.19.74.254:42801)
0806 22:44:43.806062 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.806064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.806243 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:44:43.806830 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.806831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.806840 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.806896 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.807722 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.808193 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.809153 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.809327 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.809376 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.811711 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.811712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.811712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.811719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.813063 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.813483 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.813485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.813486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.813544 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.813812 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.813813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.813813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.813933 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.814008 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.814289 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.814369 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:44:43.814656 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.805975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:43693)
0806 22:44:43.806139 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.806178 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.806745 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.806947 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.806950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.807468 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:43.807658 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.807662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.808264 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.808265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.809084 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.809086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.809393 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.809394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.809495 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.809936 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:43.809945 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:43.811627 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:43.813165 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.813172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.813175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.813396 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.813628 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.813629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.813630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.813745 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.813999 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:43.814007 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:43.814150 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.814250 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.814407 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260806 22:44:43.815137 1873 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42801
I20260806 22:44:43.815231 1873 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:43.815413 1873 heartbeater.cc:511] Master 127.19.74.254:42801 requested a full tablet report, sending...
I20260806 22:44:43.815753 1399 ts_manager.cc:194] Registered new tserver with Master: daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
I20260806 22:44:43.816128 1399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:43693
I20260806 22:44:43.822000 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:44:43.829281 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.823351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.19.74.254:42801)
0806 22:44:43.823403 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.823405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.823520 (+ 115us) server_negotiation.cc:407] Connection header received
0806 22:44:43.823585 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.823586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.823605 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.823667 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.824244 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.824743 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.825480 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.825671 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.825738 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.826906 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.826907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.826908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.826915 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.828089 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.828475 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.828476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.828477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.828531 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.828743 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.828744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.828744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.828884 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.828990 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.829104 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.829180 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:44:43.830721 1399 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:43.832214 1399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:43.838364 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.834354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:43867 (local address 127.0.0.1:43582)
0806 22:44:43.834695 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.834737 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.834807 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.835065 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.835068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.835071 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.835143 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.835144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.836513 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.836514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.837350 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.837353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.838036 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.838037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.838115 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.838148 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.838241 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.838291 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":77,"thread_start_us":74,"threads_started":1}
I20260806 22:44:43.838383 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.834458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43582 (local address 127.19.74.194:43867)
0806 22:44:43.834647 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.834653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.834888 (+ 235us) server_negotiation.cc:407] Connection header received
0806 22:44:43.834933 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.834935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.834954 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.835053 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.835208 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.836377 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.837445 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.837970 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.838032 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.838091 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.838258 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.838302 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260806 22:44:43.838829 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.834558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43235 (local address 127.0.0.1:49276)
0806 22:44:43.834725 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.834737 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.834803 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.835544 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.835545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.835547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.835583 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.835585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.836442 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.836443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.837733 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.837735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.838558 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.838558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.838653 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.838658 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.838717 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.838760 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":77,"thread_start_us":32,"threads_started":1}
I20260806 22:44:43.838842 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.834735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49276 (local address 127.19.74.195:43235)
0806 22:44:43.835381 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.835385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.835394 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:44:43.835429 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.835430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.835441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.835650 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.835700 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.836349 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.837834 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.838488 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.838557 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.838640 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.838731 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.838767 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":93,"threads_started":1}
I20260806 22:44:43.839450 1667 tablet_service.cc:1467] Processing CreateTablet for tablet b701b816d10f4684a98e98778b987887 (DEFAULT_TABLE table=test-table [id=73ce89053f00410580e568cd798c2e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.839704 1881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.834362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:46523 (local address 127.0.0.1:33894)
0806 22:44:43.834695 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.834737 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.834803 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.835192 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.835193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.835195 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.835248 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.835249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.836496 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.836496 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.837732 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.837735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.839464 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.839466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.839529 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.839533 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.839587 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.839637 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":27,"thread_start_us":55,"threads_started":1}
I20260806 22:44:43.839742 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.834465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33894 (local address 127.19.74.193:46523)
0806 22:44:43.834952 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.834956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.834963 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:44:43.834996 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.834997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.835008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.835104 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.835333 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.836412 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.838690 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.839374 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.839451 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.839505 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.839615 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.839655 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":87,"threads_started":1}
I20260806 22:44:43.839845 1667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b701b816d10f4684a98e98778b987887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.839774 1804 tablet_service.cc:1467] Processing CreateTablet for tablet b701b816d10f4684a98e98778b987887 (DEFAULT_TABLE table=test-table [id=73ce89053f00410580e568cd798c2e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.840123 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b701b816d10f4684a98e98778b987887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.840389 1527 tablet_service.cc:1467] Processing CreateTablet for tablet b701b816d10f4684a98e98778b987887 (DEFAULT_TABLE table=test-table [id=73ce89053f00410580e568cd798c2e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.840708 1527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b701b816d10f4684a98e98778b987887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.842463 1887 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: Bootstrap starting.
I20260806 22:44:43.843062 1887 tablet_bootstrap.cc:654] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.843456 1888 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Bootstrap starting.
I20260806 22:44:43.843674 1887 log.cc:824] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:43.843895 1890 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Bootstrap starting.
I20260806 22:44:43.844142 1888 tablet_bootstrap.cc:654] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.844493 1890 tablet_bootstrap.cc:654] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.844668 1888 log.cc:824] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:43.845010 1890 log.cc:824] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:43.845381 1888 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: No bootstrap required, opened a new log
I20260806 22:44:43.845491 1888 ts_tablet_manager.cc:1397] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:43.845708 1890 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: No bootstrap required, opened a new log
I20260806 22:44:43.845721 1887 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: No bootstrap required, opened a new log
I20260806 22:44:43.845780 1890 ts_tablet_manager.cc:1397] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:43.845780 1887 ts_tablet_manager.cc:1397] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:44:43.847000 1888 raft_consensus.cc:348] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.847141 1888 raft_consensus.cc:374] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.847204 1888 raft_consensus.cc:729] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b0dde0622c74831a930dd9098982878, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.847332 1888 consensus_queue.cc:260] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.847607 1734 heartbeater.cc:503] Master 127.19.74.254:42801 was elected leader, sending a full tablet report...
I20260806 22:44:43.847620 1888 ts_tablet_manager.cc:1428] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:43.847719 1887 raft_consensus.cc:348] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.847719 1890 raft_consensus.cc:348] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.847856 1887 raft_consensus.cc:374] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.847867 1890 raft_consensus.cc:374] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.847903 1887 raft_consensus.cc:729] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daedd8e176d64bba9b5dc8e3d7392f1b, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.847903 1890 raft_consensus.cc:729] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7304c1c03da94150b6303fd361f3be6d, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.848016 1887 consensus_queue.cc:260] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.848037 1890 consensus_queue.cc:260] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.848286 1887 ts_tablet_manager.cc:1428] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:44:43.848286 1890 ts_tablet_manager.cc:1428] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:43.848426 1873 heartbeater.cc:503] Master 127.19.74.254:42801 was elected leader, sending a full tablet report...
I20260806 22:44:43.848490 1597 heartbeater.cc:503] Master 127.19.74.254:42801 was elected leader, sending a full tablet report...
I20260806 22:44:43.882727 1895 raft_consensus.cc:482] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:43.882846 1895 raft_consensus.cc:504] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.883178 1895 leader_election.cc:288] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), 7304c1c03da94150b6303fd361f3be6d (127.19.74.193:46523)
I20260806 22:44:43.885923 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.883324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:43867 (local address 127.19.74.195:33885)
0806 22:44:43.883384 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.883396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.883442 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.883641 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.883642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.883644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.883716 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.883718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.884350 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.884351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.885097 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.885099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.885654 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.885655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.885730 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.885747 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.885803 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.885846 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":19}
I20260806 22:44:43.886009 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.883441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:33885 (local address 127.19.74.194:43867)
0806 22:44:43.883484 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.883486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.883493 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:44:43.883538 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.883539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.883555 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.883618 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.883785 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.884264 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.885189 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.885569 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.885657 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.885666 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.885876 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.885927 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:44:43.886118 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.883432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:59417 (local address 127.19.74.193:46523)
0806 22:44:43.883478 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.883480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.883642 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:44:43.883708 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.883710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.883722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.883792 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.883947 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.884397 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.885310 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.885730 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.885817 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.885825 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.885988 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.886033 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:44:43.886371 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.883382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:46523 (local address 127.19.74.195:59417)
0806 22:44:43.883567 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.883583 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.883654 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.883796 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.883797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.883799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.883875 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.883877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.884481 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.884483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.885212 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.885214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.885818 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.885819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.885898 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.885902 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.886236 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.886280 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
I20260806 22:44:43.886433 1687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b701b816d10f4684a98e98778b987887" candidate_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b0dde0622c74831a930dd9098982878" is_pre_election: true
I20260806 22:44:43.886579 1687 raft_consensus.cc:2454] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daedd8e176d64bba9b5dc8e3d7392f1b in term 0.
I20260806 22:44:43.886749 1548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b701b816d10f4684a98e98778b987887" candidate_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7304c1c03da94150b6303fd361f3be6d" is_pre_election: true
I20260806 22:44:43.886864 1758 leader_election.cc:302] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b0dde0622c74831a930dd9098982878, daedd8e176d64bba9b5dc8e3d7392f1b; no voters:
I20260806 22:44:43.886904 1548 raft_consensus.cc:2454] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daedd8e176d64bba9b5dc8e3d7392f1b in term 0.
I20260806 22:44:43.886979 1895 raft_consensus.cc:2781] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:43.887033 1895 raft_consensus.cc:482] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:43.887064 1895 raft_consensus.cc:3037] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.887773 1895 raft_consensus.cc:504] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.887919 1895 leader_election.cc:288] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [CANDIDATE]: Term 1 election: Requested vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), 7304c1c03da94150b6303fd361f3be6d (127.19.74.193:46523)
I20260806 22:44:43.888051 1687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b701b816d10f4684a98e98778b987887" candidate_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b0dde0622c74831a930dd9098982878"
I20260806 22:44:43.888105 1548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b701b816d10f4684a98e98778b987887" candidate_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7304c1c03da94150b6303fd361f3be6d"
I20260806 22:44:43.888139 1687 raft_consensus.cc:3037] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.888206 1548 raft_consensus.cc:3037] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.888873 1687 raft_consensus.cc:2454] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daedd8e176d64bba9b5dc8e3d7392f1b in term 1.
I20260806 22:44:43.889022 1758 leader_election.cc:302] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b0dde0622c74831a930dd9098982878, daedd8e176d64bba9b5dc8e3d7392f1b; no voters:
I20260806 22:44:43.889060 1548 raft_consensus.cc:2454] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daedd8e176d64bba9b5dc8e3d7392f1b in term 1.
I20260806 22:44:43.889117 1895 raft_consensus.cc:2781] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:43.889272 1895 raft_consensus.cc:686] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 LEADER]: Becoming Leader. State: Replica: daedd8e176d64bba9b5dc8e3d7392f1b, State: Running, Role: LEADER
I20260806 22:44:43.889374 1895 consensus_queue.cc:237] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:43.890172 1399 catalog_manager.cc:5495] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b reported cstate change: term changed from 0 to 1, leader changed from <none> to daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195). New cstate: current_term: 1 leader_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } health_report { overall_health: HEALTHY } } }
W20260806 22:44:43.900722 1735 tablet.cc:2369] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:44:43.901784 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.899552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.19.74.195:43235)
0806 22:44:43.899611 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.899625 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.899638 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:44:43.899704 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.899705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.899719 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.899777 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:43.899879 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.900303 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.901056 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.901244 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.901315 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.901422 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:43.901537 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:43.901613 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.901647 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.901666 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:44:43.903424 1548 raft_consensus.cc:1261] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Refusing update from remote peer daedd8e176d64bba9b5dc8e3d7392f1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:43.903437 1687 raft_consensus.cc:1261] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 1 FOLLOWER]: Refusing update from remote peer daedd8e176d64bba9b5dc8e3d7392f1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:43.903755 1895 consensus_queue.cc:1035] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:43.903916 1898 consensus_queue.cc:1035] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:43.906774 1902 mvcc.cc:205] Tried to move back new op lower bound from 7315686538866044928 to 7315686538811486208. Current Snapshot: MvccSnapshot[applied={T|T < 7315686538866044928}]
I20260806 22:44:43.907117 1903 mvcc.cc:205] Tried to move back new op lower bound from 7315686538866044928 to 7315686538811486208. Current Snapshot: MvccSnapshot[applied={T|T < 7315686538866044928}]
I20260806 22:44:43.908658 1399 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:43.908769 1399 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:43.909545 1399 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Sending DeleteTablet for 3 replicas of tablet b701b816d10f4684a98e98778b987887
I20260806 22:44:43.909947 1804 tablet_service.cc:1514] Processing DeleteTablet for tablet b701b816d10f4684a98e98778b987887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:49276
I20260806 22:44:43.909955 1527 tablet_service.cc:1514] Processing DeleteTablet for tablet b701b816d10f4684a98e98778b987887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:33894
I20260806 22:44:43.909992 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1463
I20260806 22:44:43.909981 1667 tablet_service.cc:1514] Processing DeleteTablet for tablet b701b816d10f4684a98e98778b987887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:43582
I20260806 22:44:43.914753 1912 tablet_replica.cc:330] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: stopping tablet replica
I20260806 22:44:43.914878 1912 raft_consensus.cc:2229] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:44:43.914968 1913 tablet_replica.cc:330] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: stopping tablet replica
I20260806 22:44:43.915122 1912 raft_consensus.cc:2258] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:43.915153 1913 raft_consensus.cc:2229] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:43.915264 1913 raft_consensus.cc:2258] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:43.915756 1912 ts_tablet_manager.cc:1905] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:43.915794 1913 ts_tablet_manager.cc:1905] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:43.917691 1912 ts_tablet_manager.cc:1918] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:43.917841 1912 log.cc:1196] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/b701b816d10f4684a98e98778b987887
I20260806 22:44:43.917850 1913 ts_tablet_manager.cc:1918] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:43.917915 1913 log.cc:1196] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/b701b816d10f4684a98e98778b987887
I20260806 22:44:43.918269 1913 ts_tablet_manager.cc:1939] T b701b816d10f4684a98e98778b987887 P 3b0dde0622c74831a930dd9098982878: Deleting consensus metadata
I20260806 22:44:43.918298 1912 ts_tablet_manager.cc:1939] T b701b816d10f4684a98e98778b987887 P daedd8e176d64bba9b5dc8e3d7392f1b: Deleting consensus metadata
I20260806 22:44:43.918809 1385 catalog_manager.cc:4841] TS 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867): tablet b701b816d10f4684a98e98778b987887 (table test-table [id=73ce89053f00410580e568cd798c2e5c]) successfully deleted
I20260806 22:44:43.918831 1386 catalog_manager.cc:4841] TS daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235): tablet b701b816d10f4684a98e98778b987887 (table test-table [id=73ce89053f00410580e568cd798c2e5c]) successfully deleted
W20260806 22:44:43.919387 1386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:46523 (error 108)
I20260806 22:44:43.919631 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1602
I20260806 22:44:43.920578 1881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.919750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:46523 (local address 127.0.0.1:33906)
0806 22:44:43.920494 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.920541 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:46523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720}
W20260806 22:44:43.921213 1386 catalog_manager.cc:4584] TS 7304c1c03da94150b6303fd361f3be6d (127.19.74.193:46523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b701b816d10f4684a98e98778b987887: Network error: Client connection negotiation failed: client connection to 127.19.74.193:46523: connect: Connection refused (error 111)
I20260806 22:44:43.926484 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1739
I20260806 22:44:43.932430 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1367
I20260806 22:44:43.939111 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42801
--webserver_interface=127.19.74.254
--webserver_port=33199
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:42801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:44.020859 1915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:44.021087 1915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:44.021109 1915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:44.022166 1915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:44.022199 1915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:44.022213 1915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:44.022226 1915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:44.022238 1915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:44.023869 1915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:42801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=33199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:44.024072 1915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:44.024241 1915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:44.035221 1921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.035228 1922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.035292 1924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:44.035360 1915 server_base.cc:1034] running on GCE node
I20260806 22:44:44.035733 1915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:44.036033 1915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:44.037171 1915 hybrid_clock.cc:648] HybridClock initialized: now 1786056284037153 us; error 37 us; skew 500 ppm
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:44.039764 1915 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:44:44.040235 1915 webserver.cc:460] Webserver started at http://127.19.74.254:33199/ using document root <none> and password file <none>
I20260806 22:44:44.040447 1915 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:44.040499 1915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:44.041805 1915 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:44.042423 1931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.042637 1915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:44.042743 1915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "527a9efe72ec4d228b5bdb08b0270bc6"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "dd2585eb7cd7dc750c4dc69272331351"
server_key_iv: "d3e9135de52ac62d6566550747309fca"
server_key_version: "encryptionkey@0"
I20260806 22:44:44.043126 1915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:44.067610 1915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:44.069473 1915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:44.069639 1915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:44.074002 1915 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:42801
I20260806 22:44:44.074040 1984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:42801 every 8 connection(s)
I20260806 22:44:44.074407 1915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:44.076043 1985 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:44:44.076758 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Bootstrap starting.
I20260806 22:44:44.084259 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1915
I20260806 22:44:44.084586 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:46523
--local_ip_for_outbound_sockets=127.19.74.193
--tserver_master_addrs=127.19.74.254:42801
--webserver_port=33981
--webserver_interface=127.19.74.193
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:44.085903 1985 log.cc:824] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:44.087872 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:44:44.088300 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Bootstrap complete.
I20260806 22:44:44.090678 1985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:44.090955 1985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527a9efe72ec4d228b5bdb08b0270bc6, State: Initialized, Role: FOLLOWER
I20260806 22:44:44.091130 1985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:44.091203 1985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:44.091260 1985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:44.091306 1985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:44:44.092300 1985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:44.092422 1985 leader_election.cc:302] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 527a9efe72ec4d228b5bdb08b0270bc6; no voters:
I20260806 22:44:44.092613 1985 leader_election.cc:288] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:44:44.092648 1989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:44:44.092762 1989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 2 LEADER]: Becoming Leader. State: Replica: 527a9efe72ec4d228b5bdb08b0270bc6, State: Running, Role: LEADER
I20260806 22:44:44.092896 1985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:44.092913 1989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:44.093326 1991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 527a9efe72ec4d228b5bdb08b0270bc6. Latest consensus state: current_term: 2 leader_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } } }
I20260806 22:44:44.093420 1991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:44.093659 1990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } } }
I20260806 22:44:44.093736 1990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:44.093902 1997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:44.094826 1997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:44.095124 1997 catalog_manager.cc:1259] Loaded cluster ID: 64864ccfd1d74071843f3721a9086505
I20260806 22:44:44.095177 1997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:44.095435 1997 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 22:44:44.098060 1997 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 22:44:44.099933 1997 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:44.100153 1997 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Loaded TSK: 0
I20260806 22:44:44.100366 1997 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:44:44.187501 1988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:44.187726 1988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:44.187765 1988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:44.188925 1988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:44.188985 1988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:44.189031 1988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:44:44.190565 1988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:46523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=33981
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:44.190869 1988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:44.191089 1988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:44.196223 2013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.196233 2016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:44.196441 1988 server_base.cc:1034] running on GCE node
W20260806 22:44:44.196239 2014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:44.196714 1988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:44.196959 1988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:44.198122 1988 hybrid_clock.cc:648] HybridClock initialized: now 1786056284198101 us; error 40 us; skew 500 ppm
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:44.200651 1988 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:44:44.201078 1988 webserver.cc:460] Webserver started at http://127.19.74.193:33981/ using document root <none> and password file <none>
I20260806 22:44:44.201314 1988 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:44.201390 1988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:44.202762 1988 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:44.203477 2023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.203650 1988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:44:44.203758 1988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "7304c1c03da94150b6303fd361f3be6d"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "bda80debb0b1e93fff101bc849ba78cb"
server_key_iv: "07f82c88317dc808f2f3a753c1b73b3d"
server_key_version: "encryptionkey@0"
I20260806 22:44:44.204064 1988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:44.214802 1988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:44.216861 1988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:44.217010 1988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:44.217235 1988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:44.217653 2030 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:44:44.219534 1988 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:44:44.219589 1988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:44:44.219635 1988 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:44:44.220266 1988 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:44:44.220317 1988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:44:44.220392 2030 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Bootstrap starting.
I20260806 22:44:44.227686 1988 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:46523
I20260806 22:44:44.228055 1988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:44.230866 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 1988
I20260806 22:44:44.231139 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:43867
--local_ip_for_outbound_sockets=127.19.74.194
--tserver_master_addrs=127.19.74.254:42801
--webserver_port=33945
--webserver_interface=127.19.74.194
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:44.232190 2030 log.cc:824] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:44.232530 2141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:46523 every 8 connection(s)
I20260806 22:44:44.234709 2030 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:44:44.235072 2030 tablet_bootstrap.cc:492] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Bootstrap complete.
I20260806 22:44:44.236250 2030 ts_tablet_manager.cc:1397] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260806 22:44:44.237229 2030 raft_consensus.cc:348] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:44.237459 2030 raft_consensus.cc:729] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7304c1c03da94150b6303fd361f3be6d, State: Initialized, Role: FOLLOWER
I20260806 22:44:44.237634 2030 consensus_queue.cc:260] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } }
I20260806 22:44:44.237931 2030 ts_tablet_manager.cc:1428] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:44.244289 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.228815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:38923)
0806 22:44:44.229043 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.229096 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.229500 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.233354 (+ 3854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:44.233358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:44.233708 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:44.233950 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.233957 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.235672 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.235673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.236884 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.236888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.238743 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.238744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.238874 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.239419 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:44.239428 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:44.240832 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:44.242736 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:44.242743 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:44.242745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:44.242977 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:44.243239 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:44.243240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:44.243241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:44.243345 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:44.243695 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:44.243701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:44.243830 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 22:44:44.243926 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:44:44.244097 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":99,"threads_started":1}
I20260806 22:44:44.244289 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.228890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:38923 (local address 127.19.74.254:42801)
0806 22:44:44.232875 (+ 3985us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.232884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.232929 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:44:44.233061 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.233068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.233150 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.233294 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:44.234087 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.235545 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.236999 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.237559 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.238777 (+ 1218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.240957 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:44.240966 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:44.240966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:44.240976 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:44.242650 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:44.243054 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:44.243057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:44.243058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:44.243150 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:44.243406 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:44.243407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:44.243407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:44.243619 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:44.243701 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.243992 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.244123 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3932,"thread_start_us":95,"threads_started":1}
I20260806 22:44:44.245352 2142 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42801
I20260806 22:44:44.245458 2142 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:44.245707 2142 heartbeater.cc:511] Master 127.19.74.254:42801 requested a full tablet report, sending...
I20260806 22:44:44.246441 1948 ts_manager.cc:194] Registered new tserver with Master: 7304c1c03da94150b6303fd361f3be6d (127.19.74.193:46523)
I20260806 22:44:44.247614 1948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:38923
I20260806 22:44:44.248562 2142 heartbeater.cc:503] Master 127.19.74.254:42801 was elected leader, sending a full tablet report...
I20260806 22:44:44.257512 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.247069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:46523 (local address 127.0.0.1:33916)
0806 22:44:44.250719 (+ 3650us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.250750 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.250806 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.254857 (+ 4051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:44.254860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:44.254862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:44.254927 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.254929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.255737 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.255739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.256575 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.256578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.257199 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.257201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.257283 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.257301 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 22:44:44.257387 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:44:44.257440 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3613,"thread_start_us":89,"threads_started":1}
I20260806 22:44:44.257512 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.247320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33916 (local address 127.19.74.193:46523)
0806 22:44:44.254713 (+ 7393us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.254717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.254725 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:44.254752 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.254753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.254764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.254849 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:44.255014 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.255653 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.256668 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.257116 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.257198 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.257254 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.257405 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.257444 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7365,"thread_start_us":78,"threads_started":1}
I20260806 22:44:44.258083 2056 tablet_service.cc:1514] Processing DeleteTablet for tablet b701b816d10f4684a98e98778b987887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:33916
I20260806 22:44:44.258575 2151 tablet_replica.cc:330] stopping tablet replica
I20260806 22:44:44.258740 2151 raft_consensus.cc:2229] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:44.258834 2151 raft_consensus.cc:2258] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:44.259143 2151 ts_tablet_manager.cc:1905] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:44.261487 2151 ts_tablet_manager.cc:1918] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:44.261575 2151 log.cc:1196] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/b701b816d10f4684a98e98778b987887
I20260806 22:44:44.261793 2151 ts_tablet_manager.cc:1939] T b701b816d10f4684a98e98778b987887 P 7304c1c03da94150b6303fd361f3be6d: Deleting consensus metadata
I20260806 22:44:44.262272 1935 catalog_manager.cc:4841] TS 7304c1c03da94150b6303fd361f3be6d (127.19.74.193:46523): tablet b701b816d10f4684a98e98778b987887 (table test-table [id=73ce89053f00410580e568cd798c2e5c]) successfully deleted
W20260806 22:44:44.324288 2147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:44.324525 2147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:44.324549 2147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:44.325733 2147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:44.325783 2147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:44.325812 2147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:44:44.327401 2147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:43867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=33945
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:44.327657 2147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:44.327889 2147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:44.332849 2156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.332986 2157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.332988 2159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:44.333009 2147 server_base.cc:1034] running on GCE node
I20260806 22:44:44.333568 2147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:44.333849 2147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:44.335026 2147 hybrid_clock.cc:648] HybridClock initialized: now 1786056284335007 us; error 45 us; skew 500 ppm
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:44.337523 2147 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:44:44.337984 2147 webserver.cc:460] Webserver started at http://127.19.74.194:33945/ using document root <none> and password file <none>
I20260806 22:44:44.338217 2147 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:44.338299 2147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:44.339797 2147 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:44.340379 2166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.340529 2147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:44.340632 2147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "3b0dde0622c74831a930dd9098982878"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "131b219050e24aacdf4646dbe3f6d6e9"
server_key_iv: "bd3ab8d33f919bb0c78b533fad4cbaa7"
server_key_version: "encryptionkey@0"
I20260806 22:44:44.340937 2147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:44.353111 2147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:44.354892 2147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:44.355042 2147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:44.355250 2147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:44.355546 2147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:44.355595 2147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.355639 2147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:44.355669 2147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.361721 2147 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:43867
I20260806 22:44:44.361766 2282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:43867 every 8 connection(s)
I20260806 22:44:44.362097 2147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:44.371018 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.362795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:39341 (local address 127.19.74.254:42801)
0806 22:44:44.362855 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.362857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.363082 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:44:44.363601 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.363602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.363618 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.363676 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:44.364601 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.365117 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.366152 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.366385 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.366468 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.368432 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:44.368433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:44.368433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:44.368440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:44.369623 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:44.370034 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:44.370035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:44.370036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:44.370096 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:44.370331 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:44.370332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:44.370333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:44.370446 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:44.370518 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.370841 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.370934 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:44:44.371193 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.362801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:39341)
0806 22:44:44.362968 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.363003 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.363497 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.363747 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:44.363751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:44.364314 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:44.364502 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.364506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.365226 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.365227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.366042 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.366045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.366481 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.366482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.366643 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.367072 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:44.367080 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:44.368331 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:44.369717 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:44.369725 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:44.369727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:44.369948 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:44.370166 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:44.370167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:44.370167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:44.370267 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:44.370509 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:44.370516 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:44.370716 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 22:44:44.370818 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:44:44.370966 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260806 22:44:44.371513 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 2147
I20260806 22:44:44.371759 2283 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42801
I20260806 22:44:44.371809 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:43235
--local_ip_for_outbound_sockets=127.19.74.195
--tserver_master_addrs=127.19.74.254:42801
--webserver_port=38703
--webserver_interface=127.19.74.195
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:44.371862 2283 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:44.372049 2283 heartbeater.cc:511] Master 127.19.74.254:42801 requested a full tablet report, sending...
I20260806 22:44:44.372478 1948 ts_manager.cc:194] Registered new tserver with Master: 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867)
I20260806 22:44:44.372854 1948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:39341
W20260806 22:44:44.455189 2287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:44.455394 2287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:44.455418 2287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:44.456532 2287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:44.456583 2287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:44.456643 2287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:44:44.458057 2287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:43235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=38703
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:44.458285 2287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:44.458505 2287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:44.463471 2292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.463551 2295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.463507 2293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:44.463564 2287 server_base.cc:1034] running on GCE node
I20260806 22:44:44.463881 2287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:44.464109 2287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:44.465253 2287 hybrid_clock.cc:648] HybridClock initialized: now 1786056284465243 us; error 28 us; skew 500 ppm
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:44.467747 2287 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:44:44.468173 2287 webserver.cc:460] Webserver started at http://127.19.74.195:38703/ using document root <none> and password file <none>
I20260806 22:44:44.468386 2287 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:44.468461 2287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:44.469769 2287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:44.470315 2302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.470479 2287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:44.470577 2287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "daedd8e176d64bba9b5dc8e3d7392f1b"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "188bf9e01d0693112033bdab45d77073"
server_key_iv: "2ad870bb132507ca656844e3dba67109"
server_key_version: "encryptionkey@0"
I20260806 22:44:44.470929 2287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:44.481771 2287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:44.483594 2287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:44.483753 2287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:44.483979 2287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:44.484265 2287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:44.484335 2287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.484375 2287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:44.484406 2287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.490139 2287 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:43235
I20260806 22:44:44.490187 2420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:43235 every 8 connection(s)
I20260806 22:44:44.490506 2287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:44:44.497599 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 2287
I20260806 22:44:44.499935 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.491234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:54513 (local address 127.19.74.254:42801)
0806 22:44:44.491303 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.491305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.491503 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:44:44.491948 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.491949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.491957 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.492028 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:44.493149 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.493704 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.494801 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.494975 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.495029 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.496939 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:44.496940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:44.496941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:44.496947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:44.498611 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:44.498986 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:44.498988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:44.498988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:44.499051 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:44.499319 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:44.499320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:44.499320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:44.499415 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:44.499464 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.499759 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.499847 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:44:44.500121 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.491229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:54513)
0806 22:44:44.491385 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.491423 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.491868 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.492079 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:44.492082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:44.492673 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:44.492904 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.492909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.493816 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.493817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.494732 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.494735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.495041 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.495042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.495139 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.495548 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:44.495556 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:44.496859 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:44.498703 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:44.498709 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:44.498711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:44.498916 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:44.499114 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:44.499115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:44.499115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:44.499250 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:44.499473 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:44.499479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:44.499632 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 22:44:44.499738 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:44:44.499897 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260806 22:44:44.500535 2421 heartbeater.cc:347] Connected to a master server at 127.19.74.254:42801
I20260806 22:44:44.500634 2421 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:44.500823 2421 heartbeater.cc:511] Master 127.19.74.254:42801 requested a full tablet report, sending...
I20260806 22:44:44.501235 1948 ts_manager.cc:194] Registered new tserver with Master: daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
I20260806 22:44:44.501736 1948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:54513
I20260806 22:44:44.504480 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.497980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35792 (local address 127.19.74.254:42801)
0806 22:44:44.498095 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.498096 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.498143 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:44:44.498212 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.498213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.498222 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.498278 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:44.499039 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.499758 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.500752 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.500990 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.501061 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.502285 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:44.502286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:44.502286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:44.502292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:44.503634 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:44.503922 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:44.503922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:44.503923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:44.503972 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:44.504120 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:44.504120 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:44.504120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:44.504196 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:44.504309 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.504334 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.504391 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":48,"threads_started":1}
I20260806 22:44:44.505118 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:44:44.507972 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.505438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.19.74.254:42801)
0806 22:44:44.505491 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.505493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.505586 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:44:44.505670 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.505671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.505706 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.505801 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:44.505909 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.506347 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.507159 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.507355 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.507421 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.507590 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:44.507712 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:44.507865 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.507884 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.507904 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:44:44.508224 1948 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:44.509800 1948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:44.511933 2056 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc45cbc008949bd821e79430c8e3954 (DEFAULT_TABLE table=test-table [id=6fe3f12a8cc84ed58752a043ab7e6c37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:44.512145 2056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc45cbc008949bd821e79430c8e3954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:44.513674 2030 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Bootstrap starting.
I20260806 22:44:44.514009 2030 tablet_bootstrap.cc:654] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:44.514669 2030 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: No bootstrap required, opened a new log
I20260806 22:44:44.514724 2030 ts_tablet_manager.cc:1397] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:44.514835 2030 raft_consensus.cc:348] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.514896 2030 raft_consensus.cc:374] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:44.514937 2030 raft_consensus.cc:729] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7304c1c03da94150b6303fd361f3be6d, State: Initialized, Role: FOLLOWER
I20260806 22:44:44.514987 2030 consensus_queue.cc:260] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.515089 2030 ts_tablet_manager.cc:1428] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.515354 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.511508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:43867 (local address 127.0.0.1:43586)
0806 22:44:44.511556 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.511566 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.511603 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.512057 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:44.512058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:44.512059 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:44.512102 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.512104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.513298 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.513298 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.514151 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.514155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.515156 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.515157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.515222 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.515225 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:44.515253 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 22:44:44.515294 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:44:44.515677 2428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.511619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.19.74.194:43867)
0806 22:44:44.511895 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.511899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.511911 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:44:44.511950 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.511952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.511968 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.512050 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:44.512164 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.513202 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.514254 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.515035 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.515483 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.515552 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.515578 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.515621 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":96,"threads_started":1}
I20260806 22:44:44.516333 2217 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc45cbc008949bd821e79430c8e3954 (DEFAULT_TABLE table=test-table [id=6fe3f12a8cc84ed58752a043ab7e6c37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:44.516670 2217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc45cbc008949bd821e79430c8e3954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:44.516983 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.511510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43235 (local address 127.0.0.1:49288)
0806 22:44:44.511673 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.511684 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.511739 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.513891 (+ 2152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:44.513892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:44.513894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:44.513932 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.513934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.515074 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.515075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.516085 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.516087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.516761 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.516762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.516824 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.516828 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:44.516858 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 22:44:44.516898 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":11,"thread_start_us":100,"threads_started":1}
I20260806 22:44:44.517318 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.511688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.19.74.195:43235)
0806 22:44:44.513752 (+ 2064us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.513757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.513767 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:44:44.513800 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.513802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.513818 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.513894 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:44.513987 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.514949 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.516161 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.516663 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.517112 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.517196 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.517222 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.517258 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2028,"thread_start_us":95,"threads_started":1}
I20260806 22:44:44.517907 2352 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc45cbc008949bd821e79430c8e3954 (DEFAULT_TABLE table=test-table [id=6fe3f12a8cc84ed58752a043ab7e6c37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:44.518203 2352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc45cbc008949bd821e79430c8e3954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:44.519749 2431 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: Bootstrap starting.
I20260806 22:44:44.520447 2431 tablet_bootstrap.cc:654] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:44.521060 2431 log.cc:824] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:44.521318 2433 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: Bootstrap starting.
I20260806 22:44:44.521895 2431 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: No bootstrap required, opened a new log
I20260806 22:44:44.521950 2433 tablet_bootstrap.cc:654] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:44.521970 2431 ts_tablet_manager.cc:1397] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:44.522410 2433 log.cc:824] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:44.523130 2433 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: No bootstrap required, opened a new log
I20260806 22:44:44.523200 2433 ts_tablet_manager.cc:1397] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:44.523622 2431 raft_consensus.cc:348] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.523773 2431 raft_consensus.cc:374] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:44.523826 2431 raft_consensus.cc:729] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b0dde0622c74831a930dd9098982878, State: Initialized, Role: FOLLOWER
I20260806 22:44:44.523948 2431 consensus_queue.cc:260] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.524237 2431 ts_tablet_manager.cc:1428] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:44.524255 2283 heartbeater.cc:503] Master 127.19.74.254:42801 was elected leader, sending a full tablet report...
I20260806 22:44:44.524971 2433 raft_consensus.cc:348] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.525122 2433 raft_consensus.cc:374] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:44.525187 2433 raft_consensus.cc:729] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daedd8e176d64bba9b5dc8e3d7392f1b, State: Initialized, Role: FOLLOWER
I20260806 22:44:44.525296 2433 consensus_queue.cc:260] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.525632 2433 ts_tablet_manager.cc:1428] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:44:44.525645 2421 heartbeater.cc:503] Master 127.19.74.254:42801 was elected leader, sending a full tablet report...
I20260806 22:44:44.548178 2148 raft_consensus.cc:482] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:44.548297 2148 raft_consensus.cc:504] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.548609 2148 leader_election.cc:288] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
I20260806 22:44:44.551872 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.548766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43235 (local address 127.19.74.193:32981)
0806 22:44:44.548829 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.548843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.548896 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.549044 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:44.549045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:44.549047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:44.549109 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.549111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.549851 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.549852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.550936 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.550938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.551604 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.551604 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.551676 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.551681 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:44.551732 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:44:44.551781 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:44:44.551963 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.548831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:32981 (local address 127.19.74.195:43235)
0806 22:44:44.548875 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.548877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.548889 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:44:44.548960 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.548961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.548971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.549029 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:44.549193 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.549770 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.551014 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.551515 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.551584 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.551593 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.551820 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.551889 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:44:44.552249 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.548766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:43867 (local address 127.19.74.193:34755)
0806 22:44:44.549050 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.549061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.549130 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.549272 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:44.549273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:44.549274 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:44.549578 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.549580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.550237 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.550238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.551234 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:44.551236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.551998 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.551999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:44.552069 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.552074 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:44.552149 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:44:44.552177 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":94,"thread_start_us":100,"threads_started":1}
I20260806 22:44:44.552280 2428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.548865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:34755 (local address 127.19.74.194:43867)
0806 22:44:44.548907 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.548909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.549121 (+ 212us) server_negotiation.cc:407] Connection header received
0806 22:44:44.549197 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.549198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.549208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.549263 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:44.549650 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.550160 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.551314 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.551909 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.551982 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.552004 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.552163 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.552200 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:44:44.552579 2237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc45cbc008949bd821e79430c8e3954" candidate_uuid: "7304c1c03da94150b6303fd361f3be6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b0dde0622c74831a930dd9098982878" is_pre_election: true
I20260806 22:44:44.552687 2373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc45cbc008949bd821e79430c8e3954" candidate_uuid: "7304c1c03da94150b6303fd361f3be6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" is_pre_election: true
I20260806 22:44:44.552763 2237 raft_consensus.cc:2454] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7304c1c03da94150b6303fd361f3be6d in term 0.
I20260806 22:44:44.552842 2373 raft_consensus.cc:2454] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7304c1c03da94150b6303fd361f3be6d in term 0.
I20260806 22:44:44.552995 2026 leader_election.cc:302] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b0dde0622c74831a930dd9098982878, 7304c1c03da94150b6303fd361f3be6d; no voters:
I20260806 22:44:44.553151 2148 raft_consensus.cc:2781] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:44.553211 2148 raft_consensus.cc:482] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:44.553244 2148 raft_consensus.cc:3037] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:44.553946 2148 raft_consensus.cc:504] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.554090 2148 leader_election.cc:288] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 1 election: Requested vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
I20260806 22:44:44.554267 2237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc45cbc008949bd821e79430c8e3954" candidate_uuid: "7304c1c03da94150b6303fd361f3be6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b0dde0622c74831a930dd9098982878"
I20260806 22:44:44.554292 2373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc45cbc008949bd821e79430c8e3954" candidate_uuid: "7304c1c03da94150b6303fd361f3be6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b"
I20260806 22:44:44.554342 2237 raft_consensus.cc:3037] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:44.554376 2373 raft_consensus.cc:3037] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:44.555127 2373 raft_consensus.cc:2454] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7304c1c03da94150b6303fd361f3be6d in term 1.
I20260806 22:44:44.555130 2237 raft_consensus.cc:2454] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7304c1c03da94150b6303fd361f3be6d in term 1.
I20260806 22:44:44.555326 2027 leader_election.cc:302] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7304c1c03da94150b6303fd361f3be6d, daedd8e176d64bba9b5dc8e3d7392f1b; no voters:
I20260806 22:44:44.555433 2148 raft_consensus.cc:2781] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:44.555594 2148 raft_consensus.cc:686] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 LEADER]: Becoming Leader. State: Replica: 7304c1c03da94150b6303fd361f3be6d, State: Running, Role: LEADER
I20260806 22:44:44.555694 2148 consensus_queue.cc:237] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.556483 1947 catalog_manager.cc:5495] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7304c1c03da94150b6303fd361f3be6d (127.19.74.193). New cstate: current_term: 1 leader_uuid: "7304c1c03da94150b6303fd361f3be6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } health_report { overall_health: HEALTHY } } }
I20260806 22:44:44.573376 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.570633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.19.74.193:46523)
0806 22:44:44.570677 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:44.570679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:44.570692 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:44:44.570780 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:44.570786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:44.570802 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:44.570876 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:44.570999 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.571442 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.572584 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:44.572785 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:44.572855 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:44.573032 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:44.573133 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:44.573212 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:44.573255 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:44:44.573275 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:44:44.574824 2373 raft_consensus.cc:1261] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 FOLLOWER]: Refusing update from remote peer 7304c1c03da94150b6303fd361f3be6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:44.574823 2237 raft_consensus.cc:1261] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 1 FOLLOWER]: Refusing update from remote peer 7304c1c03da94150b6303fd361f3be6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:44.575161 2439 consensus_queue.cc:1035] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:44.575292 2148 consensus_queue.cc:1035] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:44.578068 2442 mvcc.cc:205] Tried to move back new op lower bound from 7315686541616037888 to 7315686541540786176. Current Snapshot: MvccSnapshot[applied={T|T < 7315686541616037888 or (T in {7315686541616037888})}]
I20260806 22:44:44.578388 2443 mvcc.cc:205] Tried to move back new op lower bound from 7315686541616037888 to 7315686541540786176. Current Snapshot: MvccSnapshot[applied={T|T < 7315686541616037888}]
I20260806 22:44:44.579763 1948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:44.579857 1948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:44.580583 1948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Sending DeleteTablet for 3 replicas of tablet 4dc45cbc008949bd821e79430c8e3954
I20260806 22:44:44.580981 2056 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc45cbc008949bd821e79430c8e3954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:44 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:33916
I20260806 22:44:44.580960 2217 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc45cbc008949bd821e79430c8e3954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:44 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:43586
I20260806 22:44:44.581012 2352 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc45cbc008949bd821e79430c8e3954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:44 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:49288
I20260806 22:44:44.581063 2151 tablet_replica.cc:330] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: stopping tablet replica
I20260806 22:44:44.581112 2151 raft_consensus.cc:2229] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:44:44.581194 2151 raft_consensus.cc:2258] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:44.581467 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1988
I20260806 22:44:44.581480 2151 ts_tablet_manager.cc:1905] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:44.582338 2452 tablet_replica.cc:330] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: stopping tablet replica
I20260806 22:44:44.582427 2452 raft_consensus.cc:2229] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:44.582494 2452 raft_consensus.cc:2258] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:44.583065 2452 ts_tablet_manager.cc:1905] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:44.583456 2451 tablet_replica.cc:330] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: stopping tablet replica
I20260806 22:44:44.583572 2451 raft_consensus.cc:2229] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:44.583671 2451 raft_consensus.cc:2258] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:44.585038 2452 ts_tablet_manager.cc:1918] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:44.585103 2452 log.cc:1196] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/4dc45cbc008949bd821e79430c8e3954
I20260806 22:44:44.585377 2452 ts_tablet_manager.cc:1939] T 4dc45cbc008949bd821e79430c8e3954 P daedd8e176d64bba9b5dc8e3d7392f1b: Deleting consensus metadata
I20260806 22:44:44.585955 1935 catalog_manager.cc:4841] TS daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235): tablet 4dc45cbc008949bd821e79430c8e3954 (table test-table [id=6fe3f12a8cc84ed58752a043ab7e6c37]) successfully deleted
W20260806 22:44:44.589807 1935 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:46523 (error 108)
I20260806 22:44:44.589948 2451 ts_tablet_manager.cc:1905] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:44.591718 2451 ts_tablet_manager.cc:1918] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260806 22:44:44.591743 2427 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:44.590286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:46523 (local address 127.0.0.1:33946)
0806 22:44:44.590344 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.590358 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.590410 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.591688 (+ 1278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:46523: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:44:44.591807 2451 log.cc:1196] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/4dc45cbc008949bd821e79430c8e3954
I20260806 22:44:44.592118 2451 ts_tablet_manager.cc:1939] T 4dc45cbc008949bd821e79430c8e3954 P 3b0dde0622c74831a930dd9098982878: Deleting consensus metadata
W20260806 22:44:44.591964 1935 catalog_manager.cc:4584] TS 7304c1c03da94150b6303fd361f3be6d (127.19.74.193:46523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4dc45cbc008949bd821e79430c8e3954: Network error: Client connection negotiation failed: client connection to 127.19.74.193:46523: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:44:44.592622 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 2147
I20260806 22:44:44.592687 1934 catalog_manager.cc:4841] TS 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867): tablet 4dc45cbc008949bd821e79430c8e3954 (table test-table [id=6fe3f12a8cc84ed58752a043ab7e6c37]) successfully deleted
I20260806 22:44:44.598601 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 2287
I20260806 22:44:44.604562 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 1915
I20260806 22:44:44.611069 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42801
--webserver_interface=127.19.74.254
--webserver_port=33199
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:42801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:44.692898 2454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:44.693100 2454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:44.693123 2454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:44.694164 2454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:44.694195 2454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:44.694208 2454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:44.694221 2454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:44.694231 2454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:44.695580 2454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:42801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=33199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:44.695758 2454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:44.695915 2454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:44.700480 2460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.700532 2462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.700572 2459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:44.700852 2454 server_base.cc:1034] running on GCE node
I20260806 22:44:44.701102 2454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:44.701378 2454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:44.702533 2454 hybrid_clock.cc:648] HybridClock initialized: now 1786056284702521 us; error 35 us; skew 500 ppm
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:44.704910 2454 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:44:44.705338 2454 webserver.cc:460] Webserver started at http://127.19.74.254:33199/ using document root <none> and password file <none>
I20260806 22:44:44.705557 2454 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:44.705636 2454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:44.706972 2454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:44.707620 2469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.707784 2454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:44.707888 2454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "527a9efe72ec4d228b5bdb08b0270bc6"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "dd2585eb7cd7dc750c4dc69272331351"
server_key_iv: "d3e9135de52ac62d6566550747309fca"
server_key_version: "encryptionkey@0"
I20260806 22:44:44.708220 2454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:44.723546 2454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:44.725371 2454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:44.725521 2454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:44.729601 2454 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:42801
I20260806 22:44:44.729638 2524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:42801 every 8 connection(s)
I20260806 22:44:44.730057 2454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:44.731688 2525 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:44:44.732331 2525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Bootstrap starting.
I20260806 22:44:44.736447 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 2454
I20260806 22:44:44.736770 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:46523
--local_ip_for_outbound_sockets=127.19.74.193
--tserver_master_addrs=127.19.74.254:42801
--webserver_port=33981
--webserver_interface=127.19.74.193
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:44.741684 2525 log.cc:824] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:44.743930 2525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:44:44.744267 2525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6: Bootstrap complete.
I20260806 22:44:44.746531 2525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:44.746810 2525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527a9efe72ec4d228b5bdb08b0270bc6, State: Initialized, Role: FOLLOWER
I20260806 22:44:44.746982 2525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:44.747066 2525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:44.747123 2525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:44.747171 2525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:44:44.748157 2525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:44.748263 2525 leader_election.cc:302] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 527a9efe72ec4d228b5bdb08b0270bc6; no voters:
I20260806 22:44:44.748451 2525 leader_election.cc:288] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:44:44.748489 2529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:44:44.748620 2529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [term 3 LEADER]: Becoming Leader. State: Replica: 527a9efe72ec4d228b5bdb08b0270bc6, State: Running, Role: LEADER
I20260806 22:44:44.748720 2525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:44.748708 2529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } }
I20260806 22:44:44.748981 2529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 527a9efe72ec4d228b5bdb08b0270bc6. Latest consensus state: current_term: 3 leader_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } } }
I20260806 22:44:44.749061 2529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:44.748986 2530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527a9efe72ec4d228b5bdb08b0270bc6" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42801 } } }
I20260806 22:44:44.749220 2530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 527a9efe72ec4d228b5bdb08b0270bc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:44.749284 2533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:44.750164 2533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:44.750285 2533 catalog_manager.cc:1259] Loaded cluster ID: 64864ccfd1d74071843f3721a9086505
I20260806 22:44:44.750327 2533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 22:44:44.750698 2533 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f37ae40e9ad google::LogMessage::Fail() at ??:0
@ 0x7f37ae41188a google::LogMessage::SendToLog() at ??:0
@ 0x7f37ae40e4de google::LogMessage::Flush() at ??:0
@ 0x7f37ae40fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f37b63330f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f37b633c8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f37b02dc0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f37b02d1954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f37b11356db start_thread at ??:0
@ 0x7f37acf0c61f clone at ??:0
W20260806 22:44:44.836536 2527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:44.836731 2527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:44.836753 2527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:44.837920 2527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:44.837962 2527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:44.838006 2527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:44:44.839562 2527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:46523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=33981
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:44.839807 2527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:44.840040 2527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:44.845577 2565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.845511 2563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:44.845546 2562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:44.845794 2527 server_base.cc:1034] running on GCE node
I20260806 22:44:44.846009 2527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:44.846244 2527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:44.847429 2527 hybrid_clock.cc:648] HybridClock initialized: now 1786056284847416 us; error 29 us; skew 500 ppm
Aug 06 22:44:44 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:44.849920 2527 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:44:44.850319 2527 webserver.cc:460] Webserver started at http://127.19.74.193:33981/ using document root <none> and password file <none>
I20260806 22:44:44.850533 2527 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:44.850603 2527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:44.851920 2527 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:44.852766 2573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:44.852938 2527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:44.853046 2527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "7304c1c03da94150b6303fd361f3be6d"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "bda80debb0b1e93fff101bc849ba78cb"
server_key_iv: "07f82c88317dc808f2f3a753c1b73b3d"
server_key_version: "encryptionkey@0"
I20260806 22:44:44.853366 2527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:44.879451 2527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:44.881903 2527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:44.882066 2527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:44.882313 2527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:44.882838 2580 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:44:44.884826 2527 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:44:44.884886 2527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:44:44.884929 2527 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:44:44.885571 2527 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:44:44.885625 2527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:44:44.885676 2580 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Bootstrap starting.
I20260806 22:44:44.887603 2580 log.cc:824] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:44.890658 2580 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:44:44.890957 2580 tablet_bootstrap.cc:492] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Bootstrap complete.
I20260806 22:44:44.891117 2580 ts_tablet_manager.cc:1397] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 22:44:44.892853 2527 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:46523
I20260806 22:44:44.893242 2527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:44.892936 2580 raft_consensus.cc:348] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.893335 2580 raft_consensus.cc:729] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7304c1c03da94150b6303fd361f3be6d, State: Initialized, Role: FOLLOWER
I20260806 22:44:44.893515 2580 consensus_queue.cc:260] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:44.892889 2690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:46523 every 8 connection(s)
I20260806 22:44:44.893911 2580 ts_tablet_manager.cc:1428] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:44.903282 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 2527
I20260806 22:44:44.903561 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:43867
--local_ip_for_outbound_sockets=127.19.74.194
--tserver_master_addrs=127.19.74.254:42801
--webserver_port=33945
--webserver_interface=127.19.74.194
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:44.932936 2694 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:44.893933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:33253)
0806 22:44:44.894101 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.894133 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:44.894289 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:44.932694 (+ 38405us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
W20260806 22:44:44.933161 2691 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:42801 (0 consecutive failures): Network error: Failed to ping master at 127.19.74.254:42801: Client connection negotiation failed: client connection to 127.19.74.254:42801: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:44:44.933563 2694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.933431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:50181)
0806 22:44:44.933486 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.933531 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:44:44.933846 2694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:44.933768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:51873)
0806 22:44:44.933806 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:44.933827 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 22:44:44.933948 2691 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:44:44.992640 2695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:44.992861 2695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:44.992900 2695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:44.994019 2695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:44.994082 2695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:44.994127 2695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:44:44.995558 2695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:43867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=33945
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:44.995806 2695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:44.996030 2695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:45.001138 2701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.001144 2702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:45.001160 2695 server_base.cc:1034] running on GCE node
W20260806 22:44:45.001144 2704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:45.001696 2695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:45.001896 2695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:45.003027 2695 hybrid_clock.cc:648] HybridClock initialized: now 1786056285003017 us; error 27 us; skew 500 ppm
Aug 06 22:44:45 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:45.005642 2695 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:44:45.006124 2695 webserver.cc:460] Webserver started at http://127.19.74.194:33945/ using document root <none> and password file <none>
I20260806 22:44:45.006352 2695 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:45.006415 2695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:45.007845 2695 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:45.008435 2711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.008627 2695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:45.008725 2695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "3b0dde0622c74831a930dd9098982878"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "131b219050e24aacdf4646dbe3f6d6e9"
server_key_iv: "bd3ab8d33f919bb0c78b533fad4cbaa7"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.009056 2695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:45.028364 2695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:45.030267 2695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:45.030421 2695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:45.030717 2695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:45.031028 2695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:45.031082 2695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.031122 2695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:45.031149 2695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.037184 2695 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:43867
I20260806 22:44:45.037235 2830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:43867 every 8 connection(s)
I20260806 22:44:45.037591 2695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:45.038724 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.038265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:42807)
0806 22:44:45.038421 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.038505 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
W20260806 22:44:45.039122 2831 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:42801 (0 consecutive failures): Network error: Failed to ping master at 127.19.74.254:42801: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
I20260806 22:44:45.039405 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.039304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:46411)
0806 22:44:45.039352 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.039378 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:44:45.039685 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.039587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:48581)
0806 22:44:45.039640 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.039662 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 22:44:45.039783 2831 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:44:45.040122 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 2695
I20260806 22:44:45.040342 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:43235
--local_ip_for_outbound_sockets=127.19.74.195
--tserver_master_addrs=127.19.74.254:42801
--webserver_port=38703
--webserver_interface=127.19.74.195
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:45.123435 2834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:45.123638 2834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:45.123662 2834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:45.124819 2834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:45.124861 2834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:45.124910 2834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:44:45.126421 2834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:43235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=38703
--tserver_master_addrs=127.19.74.254:42801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:44:45.126689 2834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:45.126917 2834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:45.131750 2841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.131757 2843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.131769 2839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:45.131875 2834 server_base.cc:1034] running on GCE node
I20260806 22:44:45.132180 2834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:45.132474 2834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:45.133649 2834 hybrid_clock.cc:648] HybridClock initialized: now 1786056285133642 us; error 33 us; skew 500 ppm
Aug 06 22:44:45 dist-test-slave-pgkr krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:45.136097 2834 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:44:45.136575 2834 webserver.cc:460] Webserver started at http://127.19.74.195:38703/ using document root <none> and password file <none>
I20260806 22:44:45.136822 2834 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:45.136889 2834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:45.138201 2834 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:45.138827 2851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.138991 2834 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:45.139094 2834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "daedd8e176d64bba9b5dc8e3d7392f1b"
format_stamp: "Formatted at 2026-08-06 22:44:43 on dist-test-slave-pgkr"
server_key: "188bf9e01d0693112033bdab45d77073"
server_key_iv: "2ad870bb132507ca656844e3dba67109"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.139436 2834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:45.158389 2834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:45.160274 2834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:45.160441 2834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:45.160681 2834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:45.160998 2834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:45.161048 2834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.161082 2834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:45.161119 2834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.167102 2834 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:43235
I20260806 22:44:45.167157 2969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:43235 every 8 connection(s)
I20260806 22:44:45.167486 2834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:45.168527 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.168147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:54349)
0806 22:44:45.168306 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.168384 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
W20260806 22:44:45.168959 2970 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:42801 (0 consecutive failures): Network error: Failed to ping master at 127.19.74.254:42801: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
I20260806 22:44:45.169250 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.169165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:53915)
0806 22:44:45.169207 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.169233 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:44:45.169490 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.169443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:35797)
0806 22:44:45.169462 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.169479 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260806 22:44:45.169602 2970 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:44:45.176154 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 2834
I20260806 22:44:45.934531 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.934244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:44599)
0806 22:44:45.934435 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.934469 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":56,"threads_started":1}
I20260806 22:44:46.040287 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.040069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:33907)
0806 22:44:46.040202 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.040236 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260806 22:44:46.170116 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.169917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:53221)
0806 22:44:46.170034 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.170067 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260806 22:44:46.673422 2978 raft_consensus.cc:482] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:46.673544 2978 raft_consensus.cc:504] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:46.673977 2978 leader_election.cc:288] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
Aug 06 22:44:46 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.194@KRBTEST.COM
Aug 06 22:44:46 dist-test-slave-pgkr krb5kdc[1354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056284, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.195@KRBTEST.COM
I20260806 22:44:46.684752 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.674344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:36573 (local address 127.19.74.194:43867)
0806 22:44:46.674483 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:46.674491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:46.674521 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:44:46.675118 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:46.675120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:46.675175 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:46.675321 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:46.676096 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.676983 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.677981 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.678380 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.678517 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.680995 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:46.680999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:46.681000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:46.681010 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:46.683020 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:46.683434 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:46.683437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:46.683437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:46.683533 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:46.683827 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:46.683828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:46.683828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:46.684075 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:46.684183 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.684478 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.684623 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 22:44:46.684744 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.674145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:43235 (local address 127.19.74.193:60923)
0806 22:44:46.674400 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.674409 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:46.674444 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:46.675445 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:46.675446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:46.675742 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:46.676132 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.676134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.677213 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.677214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.677879 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.677881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.678487 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.678488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.678639 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.679233 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:46.679243 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:46.681269 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:46.683404 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:46.683405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:46.683406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:46.683597 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:46.683849 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:46.683850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:46.683850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:46.683914 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:46.684320 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:46.684321 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:46.684366 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.684469 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.684628 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260806 22:44:46.684773 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.674289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:60923 (local address 127.19.74.195:43235)
0806 22:44:46.674427 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:46.674441 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:46.674467 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:44:46.675107 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:46.675112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:46.675180 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:46.675321 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:46.676259 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.677115 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.677969 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.678380 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.678515 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.681378 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:46.681382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:46.681382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:46.681393 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:46.683322 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:46.683683 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:46.683686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:46.683686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:46.683777 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:46.683981 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:46.683982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:46.683982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:46.684239 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:46.684325 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.684548 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.684676 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
I20260806 22:44:46.684849 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.674134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:43867 (local address 127.19.74.193:36573)
0806 22:44:46.674247 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.674258 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:46.674315 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:46.675392 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:46.675394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:46.675742 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:46.675978 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.675984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.677098 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.677099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.677879 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.677880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.678496 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.678497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.678636 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.679295 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:46.679300 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:46.680863 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:46.683106 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:46.683112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:46.683113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:46.683340 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:46.683637 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:46.683637 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:46.683638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:46.683749 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:46.684163 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:46.684170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:46.684300 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.684405 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.684710 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":149,"thread_start_us":56,"threads_started":1}
W20260806 22:44:46.685688 2577 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
W20260806 22:44:46.685827 2576 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
I20260806 22:44:46.685904 2576 leader_election.cc:302] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7304c1c03da94150b6303fd361f3be6d; no voters: 3b0dde0622c74831a930dd9098982878, daedd8e176d64bba9b5dc8e3d7392f1b
I20260806 22:44:46.686012 2978 raft_consensus.cc:2726] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:44:46.935117 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.934978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:51893)
0806 22:44:46.935029 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.935060 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:44:47.040948 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:47.040740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:52515)
0806 22:44:47.040856 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:47.040902 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260806 22:44:47.170837 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:47.170573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:34387)
0806 22:44:47.170752 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:47.170796 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":123,"threads_started":1}
I20260806 22:44:47.935757 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:47.935557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:59879)
0806 22:44:47.935679 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:47.935712 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260806 22:44:48.041781 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.041527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:60179)
0806 22:44:48.041699 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:48.041737 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":64,"threads_started":1}
I20260806 22:44:48.171489 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.171272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:40735)
0806 22:44:48.171414 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:48.171450 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260806 22:44:48.705755 2989 raft_consensus.cc:482] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:48.705863 2989 raft_consensus.cc:504] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:48.706048 2989 leader_election.cc:288] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
W20260806 22:44:48.706444 2577 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
W20260806 22:44:48.706523 2576 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
I20260806 22:44:48.706633 2577 leader_election.cc:302] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7304c1c03da94150b6303fd361f3be6d; no voters: 3b0dde0622c74831a930dd9098982878, daedd8e176d64bba9b5dc8e3d7392f1b
I20260806 22:44:48.706719 2989 raft_consensus.cc:2726] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:44:48.936599 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.936374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:58231)
0806 22:44:48.936515 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:48.936553 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260806 22:44:49.042486 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:49.042257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:44039)
0806 22:44:49.042390 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:49.042426 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260806 22:44:49.172163 2992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:49.171955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:34733)
0806 22:44:49.172077 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:49.172111 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260806 22:44:49.937426 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:49.937146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:56433)
0806 22:44:49.937311 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:49.937377 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":64,"threads_started":1}
I20260806 22:44:50.043175 2994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:50.042983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:54197)
0806 22:44:50.043100 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:50.043136 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260806 22:44:50.172967 2995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:50.172756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:32809)
0806 22:44:50.172889 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:50.172930 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260806 22:44:50.536367 2996 raft_consensus.cc:482] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:50.536481 2996 raft_consensus.cc:504] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:50.536728 2996 leader_election.cc:288] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
W20260806 22:44:50.537125 2577 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
W20260806 22:44:50.537236 2576 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
I20260806 22:44:50.537312 2576 leader_election.cc:302] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7304c1c03da94150b6303fd361f3be6d; no voters: 3b0dde0622c74831a930dd9098982878, daedd8e176d64bba9b5dc8e3d7392f1b
I20260806 22:44:50.537426 2996 raft_consensus.cc:2726] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:44:50.938127 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:50.937908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:43503)
0806 22:44:50.938039 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:50.938071 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260806 22:44:51.043895 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:51.043672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:53145)
0806 22:44:51.043810 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:51.043851 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260806 22:44:51.173666 3000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:51.173466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:46995)
0806 22:44:51.173587 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:51.173623 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260806 22:44:51.938813 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:51.938560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:44667)
0806 22:44:51.938734 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:51.938766 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
I20260806 22:44:52.044612 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:52.044409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:46399)
0806 22:44:52.044530 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:52.044564 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260806 22:44:52.174373 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:52.174070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:33313)
0806 22:44:52.174299 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:52.174335 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":82,"threads_started":1}
I20260806 22:44:52.939518 3004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:52.939259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:34769)
0806 22:44:52.939434 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:52.939470 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260806 22:44:53.045272 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:53.045064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:36295)
0806 22:44:53.045191 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:53.045228 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260806 22:44:53.175143 3006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:53.174930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:40227)
0806 22:44:53.175057 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:53.175092 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260806 22:44:53.940209 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:53.939988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:46541)
0806 22:44:53.940125 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:53.940161 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260806 22:44:54.045864 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:54.045673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:37741)
0806 22:44:54.045792 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:54.045824 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260806 22:44:54.175745 3009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:54.175544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:33031)
0806 22:44:54.175669 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:54.175704 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260806 22:44:54.182281 3010 raft_consensus.cc:482] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:54.182365 3010 raft_consensus.cc:504] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:54.182561 3010 leader_election.cc:288] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
W20260806 22:44:54.182955 2577 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
W20260806 22:44:54.183032 2576 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
I20260806 22:44:54.183095 2576 leader_election.cc:302] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7304c1c03da94150b6303fd361f3be6d; no voters: 3b0dde0622c74831a930dd9098982878, daedd8e176d64bba9b5dc8e3d7392f1b
I20260806 22:44:54.183187 3010 raft_consensus.cc:2726] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:44:54.940889 3011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:54.940677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:40581)
0806 22:44:54.940810 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:54.940845 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260806 22:44:55.046561 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:55.046289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:43715)
0806 22:44:55.046441 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:55.046503 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260806 22:44:55.176349 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:55.176165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:45521)
0806 22:44:55.176278 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:55.176311 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260806 22:44:55.941592 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:55.941302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:54911)
0806 22:44:55.941506 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:55.941544 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":60,"threads_started":1}
I20260806 22:44:56.047262 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:56.047046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:38805)
0806 22:44:56.047179 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:56.047217 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260806 22:44:56.176978 3016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:56.176766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:51651)
0806 22:44:56.176901 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:56.176937 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260806 22:44:56.575405 3017 raft_consensus.cc:482] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:56.575525 3017 raft_consensus.cc:504] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0dde0622c74831a930dd9098982878" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 43867 } } peers { permanent_uuid: "daedd8e176d64bba9b5dc8e3d7392f1b" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 43235 } } peers { permanent_uuid: "7304c1c03da94150b6303fd361f3be6d" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 46523 } }
I20260806 22:44:56.575735 3017 leader_election.cc:288] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867), daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235)
W20260806 22:44:56.576288 2577 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer daedd8e176d64bba9b5dc8e3d7392f1b (127.19.74.195:43235): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
W20260806 22:44:56.576397 2576 leader_election.cc:341] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b0dde0622c74831a930dd9098982878 (127.19.74.194:43867): Not found: Tablet not found: 4dc45cbc008949bd821e79430c8e3954
I20260806 22:44:56.576483 2576 leader_election.cc:302] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7304c1c03da94150b6303fd361f3be6d; no voters: 3b0dde0622c74831a930dd9098982878, daedd8e176d64bba9b5dc8e3d7392f1b
I20260806 22:44:56.576597 3017 raft_consensus.cc:2726] T 4dc45cbc008949bd821e79430c8e3954 P 7304c1c03da94150b6303fd361f3be6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:44:56.942379 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:56.942156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:38007)
0806 22:44:56.942297 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:56.942332 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260806 22:44:57.047916 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:57.047712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:51257)
0806 22:44:57.047844 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:57.047877 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260806 22:44:57.177645 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:57.177421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:37599)
0806 22:44:57.177560 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:57.177598 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260806 22:44:57.943081 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:57.942859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:52265)
0806 22:44:57.942988 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:57.943028 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260806 22:44:58.048714 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.048499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:55939)
0806 22:44:58.048632 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:58.048670 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260806 22:44:58.178306 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.178122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:43813)
0806 22:44:58.178235 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:58.178268 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260806 22:44:58.943702 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.943504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:54117)
0806 22:44:58.943620 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:58.943655 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260806 22:44:59.049460 3025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:59.049234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:47421)
0806 22:44:59.049374 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:59.049414 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260806 22:44:59.179028 3026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:59.178808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:55699)
0806 22:44:59.178952 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:59.178985 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260806 22:44:59.944424 3027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:59.944205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.193:40089)
0806 22:44:59.944330 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:59.944361 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260806 22:45:00.050365 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.050160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.194:52409)
0806 22:45:00.050287 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.050322 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260806 22:45:00.176800 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 2527
I20260806 22:45:00.179728 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.179542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:42801 (local address 127.19.74.195:34089)
0806 22:45:00.179661 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.179691 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260806 22:45:00.183959 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 2695
I20260806 22:45:00.189282 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 2834
2026-08-06T22:45:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18180 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:00 dist-test-slave-pgkr krb5kdc[3033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:00 dist-test-slave-pgkr krb5kdc[3033](info): set up 2 sockets
Aug 06 22:45:00 dist-test-slave-pgkr krb5kdc[3033](info): commencing operation
krb5kdc: starting...
W20260806 22:45:01.233534 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:45:01.357132 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38903
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:01.438971 3046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:01.439173 3046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:01.439194 3046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:01.440270 3046 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:45:01.440303 3046 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:45:01.440316 3046 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:45:01.440328 3046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:01.440341 3046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:01.440354 3046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:01.440374 3046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:01.440387 3046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:01.441787 3046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:01.441974 3046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:01.442138 3046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:01.447322 3051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.447350 3046 server_base.cc:1034] running on GCE node
W20260806 22:45:01.447340 3052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.447312 3054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.447821 3046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:01.448100 3046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:01.449257 3046 hybrid_clock.cc:648] HybridClock initialized: now 1786056301449250 us; error 40 us; skew 500 ppm
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:01.451880 3046 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:45:01.452304 3046 webserver.cc:460] Webserver started at http://127.19.74.254:36077/ using document root <none> and password file <none>
I20260806 22:45:01.452488 3046 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:01.452526 3046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:01.452623 3046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:01.453624 3046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "325b9f4c914d4c24af9a021d3bef614d"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "b136bf4b406649d02cc4ba91c438532a"
server_key_iv: "38961aef9755ca7d5055520274bb47e6"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.453935 3046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "325b9f4c914d4c24af9a021d3bef614d"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "b136bf4b406649d02cc4ba91c438532a"
server_key_iv: "38961aef9755ca7d5055520274bb47e6"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.455340 3046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:45:01.456064 3061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.456233 3046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:45:01.456342 3046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "325b9f4c914d4c24af9a021d3bef614d"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "b136bf4b406649d02cc4ba91c438532a"
server_key_iv: "38961aef9755ca7d5055520274bb47e6"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.456418 3046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:01.474048 3046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:01.475914 3046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:01.476074 3046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:01.480159 3046 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:38903
I20260806 22:45:01.480191 3116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:38903 every 8 connection(s)
I20260806 22:45:01.480587 3046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:01.481523 3117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:01.482061 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 3046
I20260806 22:45:01.482167 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:45:01.482389 19755 external_mini_cluster.cc:1428] Setting key 9b1c95616a4c63fa06ee90bbee127900
I20260806 22:45:01.484750 3117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Bootstrap starting.
I20260806 22:45:01.485369 3117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:01.485724 3117 log.cc:824] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:01.486402 3117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: No bootstrap required, opened a new log
I20260806 22:45:01.488658 3117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:01.488857 3117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:01.488910 3117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325b9f4c914d4c24af9a021d3bef614d, State: Initialized, Role: FOLLOWER
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:45:01.489060 3117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:01.489166 3117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:01.489198 3117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:01.489244 3117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:01.489753 3117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:01.489862 3117 leader_election.cc:302] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 325b9f4c914d4c24af9a021d3bef614d; no voters:
I20260806 22:45:01.490020 3117 leader_election.cc:288] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:01.490052 3122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:01.490164 3122 raft_consensus.cc:686] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 1 LEADER]: Becoming Leader. State: Replica: 325b9f4c914d4c24af9a021d3bef614d, State: Running, Role: LEADER
I20260806 22:45:01.490324 3117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:01.490598 3122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:01.491003 3123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "325b9f4c914d4c24af9a021d3bef614d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } } }
I20260806 22:45:01.491101 3123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:01.491448 3124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 325b9f4c914d4c24af9a021d3bef614d. Latest consensus state: current_term: 1 leader_uuid: "325b9f4c914d4c24af9a021d3bef614d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } } }
I20260806 22:45:01.491532 3124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:01.491804 3132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:01.492743 3132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:01.493609 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.482982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.19.74.254:38903)
0806 22:45:01.483157 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.483173 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.483198 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:45:01.483628 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.483634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.483675 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.483779 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.484534 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.485452 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.486245 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.486770 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.486913 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:01.489530 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.489534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.489535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:01.489546 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.491635 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:01.492113 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:01.492117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:01.492118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:01.492231 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:01.492468 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:01.492469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:01.492470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:01.492789 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.492972 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.493245 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.493387 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260806 22:45:01.494448 3132 catalog_manager.cc:1347] Generated new cluster ID: e582665784b64360b1f322694323a5a2
I20260806 22:45:01.494498 3132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:01.509189 3132 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:01.509871 3132 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:01.518247 3132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Generated new TSK 0
I20260806 22:45:01.518432 3132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:45:01.551251 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38903
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:01.634409 3144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:01.634603 3144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:01.634644 3144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:01.635618 3144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:01.635650 3144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:01.635672 3144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:45:01.637202 3144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:01.637393 3144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:01.637573 3144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:01.642570 3150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.642597 3152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.642657 3149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.642830 3144 server_base.cc:1034] running on GCE node
I20260806 22:45:01.643071 3144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:01.643335 3144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:01.644505 3144 hybrid_clock.cc:648] HybridClock initialized: now 1786056301644480 us; error 41 us; skew 500 ppm
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:01.647248 3144 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:45:01.647811 3144 webserver.cc:460] Webserver started at http://127.19.74.193:38925/ using document root <none> and password file <none>
I20260806 22:45:01.648175 3144 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:01.648254 3144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:01.648444 3144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:01.649822 3144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "41dc974f6da145f3a63842c794d027e0"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "aa7647354dafd39067c2771240e0e4e2"
server_key_iv: "b05d34fe6f97d6cb84191bc804997b59"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.650264 3144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "41dc974f6da145f3a63842c794d027e0"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "aa7647354dafd39067c2771240e0e4e2"
server_key_iv: "b05d34fe6f97d6cb84191bc804997b59"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.651757 3144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:45:01.652546 3159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.652730 3144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:01.652840 3144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "41dc974f6da145f3a63842c794d027e0"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "aa7647354dafd39067c2771240e0e4e2"
server_key_iv: "b05d34fe6f97d6cb84191bc804997b59"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.652915 3144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:01.666555 3144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:01.668434 3144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:01.668614 3144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:01.668852 3144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:01.669199 3144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:01.669255 3144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.669296 3144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:01.669325 3144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.675285 3144 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:45911
I20260806 22:45:01.675343 3277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:45911 every 8 connection(s)
I20260806 22:45:01.675659 3144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:01.676669 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 3144
I20260806 22:45:01.676784 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:01.676980 19755 external_mini_cluster.cc:1428] Setting key 805c6d1f6785f9ba4de85d386acacec8
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:45:01.685758 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.676378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:52871 (local address 127.19.74.254:38903)
0806 22:45:01.676445 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.676447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.676666 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:45:01.677326 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.677328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.677339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.677407 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.678483 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.679258 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.680322 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.680537 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.680646 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:01.682911 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.682913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.682914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:01.682921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.684331 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:01.684753 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:01.684754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:01.684754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:01.684819 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:01.685079 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:01.685080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:01.685080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:01.685204 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.685282 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.685563 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.685665 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
I20260806 22:45:01.685943 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.676333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:52871)
0806 22:45:01.676523 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.676566 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.677210 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.677490 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.677495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.678189 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.678381 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:01.678385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.679353 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.679354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:01.680214 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:01.680216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.680659 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.680660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:01.680790 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:01.681330 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:01.681340 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.682813 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.684434 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:01.684443 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:01.684445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:01.684676 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:01.684887 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:01.684888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:01.684889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:01.685004 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:01.685271 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.685279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.685424 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.685529 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.685693 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260806 22:45:01.686515 3278 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38903
I20260806 22:45:01.686658 3278 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:01.686884 3278 heartbeater.cc:511] Master 127.19.74.254:38903 requested a full tablet report, sending...
I20260806 22:45:01.687462 3079 ts_manager.cc:194] Registered new tserver with Master: 41dc974f6da145f3a63842c794d027e0 (127.19.74.193:45911)
I20260806 22:45:01.688054 3079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:52871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:45:01.703650 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38903
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:01.788053 3283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:01.788280 3283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:01.788303 3283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:01.789353 3283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:01.789388 3283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:01.789412 3283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:45:01.790931 3283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:01.791153 3283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:01.791373 3283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:01.796321 3289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.796293 3291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.796304 3288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.796686 3283 server_base.cc:1034] running on GCE node
I20260806 22:45:01.796849 3283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:01.797084 3283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:01.798224 3283 hybrid_clock.cc:648] HybridClock initialized: now 1786056301798215 us; error 31 us; skew 500 ppm
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:01.800783 3283 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:45:01.801209 3283 webserver.cc:460] Webserver started at http://127.19.74.194:40823/ using document root <none> and password file <none>
I20260806 22:45:01.801424 3283 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:01.801481 3283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:01.801647 3283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:01.802726 3283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "af444ef9234e445a8e4479a4d7f61da2"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "1ab88d4df3db650a2c848c9d9c7b65cb"
server_key_iv: "7906755dce9569eaedf3805a55ea907d"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.803079 3283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "af444ef9234e445a8e4479a4d7f61da2"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "1ab88d4df3db650a2c848c9d9c7b65cb"
server_key_iv: "7906755dce9569eaedf3805a55ea907d"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.804388 3283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:01.805117 3298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.805290 3283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:45:01.805398 3283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "af444ef9234e445a8e4479a4d7f61da2"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "1ab88d4df3db650a2c848c9d9c7b65cb"
server_key_iv: "7906755dce9569eaedf3805a55ea907d"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.805469 3283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:01.815089 3283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:01.816699 3283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:01.816851 3283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:01.817049 3283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:01.817350 3283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:01.817402 3283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.817440 3283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:01.817467 3283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.823225 3283 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:39563
I20260806 22:45:01.823271 3414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:39563 every 8 connection(s)
I20260806 22:45:01.823575 3283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:01.828598 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 3283
I20260806 22:45:01.828722 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:01.828938 19755 external_mini_cluster.cc:1428] Setting key 3092a767d9f14f2006aea6b7b6514fe1
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:45:01.833047 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.824211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:37449 (local address 127.19.74.254:38903)
0806 22:45:01.824255 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.824257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.824442 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:45:01.824901 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.824903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.824913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.824965 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.825860 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.826341 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.827330 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.827541 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.827593 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:01.829778 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.829779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.829779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:01.829788 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.831546 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:01.832078 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:01.832079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:01.832079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:01.832135 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:01.832434 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:01.832434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:01.832435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:01.832537 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.832589 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.832915 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.832981 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:45:01.833431 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.824193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:37449)
0806 22:45:01.824324 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.824357 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.824816 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.825015 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.825019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.825585 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.825780 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:01.825784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.826421 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.826422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:01.827255 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:01.827257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.827616 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.827616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:01.827718 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:01.828145 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:01.828153 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.829695 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.831645 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:01.831653 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:01.831656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:01.831996 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:01.832203 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:01.832203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:01.832204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:01.832367 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:01.832601 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.832608 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.832777 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.832897 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.833097 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260806 22:45:01.833977 3415 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38903
I20260806 22:45:01.834107 3415 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:01.834299 3415 heartbeater.cc:511] Master 127.19.74.254:38903 requested a full tablet report, sending...
I20260806 22:45:01.834734 3079 ts_manager.cc:194] Registered new tserver with Master: af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563)
I20260806 22:45:01.835116 3079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:37449
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:45:01.856942 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38903
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:01.940397 3420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:01.940591 3420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:01.940616 3420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:01.941668 3420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:01.941699 3420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:01.941721 3420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:45:01.943071 3420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:01.943249 3420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:01.943423 3420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:01.948385 3427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.948408 3426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.948622 3420 server_base.cc:1034] running on GCE node
W20260806 22:45:01.948385 3429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.948880 3420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:01.949128 3420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:01.950286 3420 hybrid_clock.cc:648] HybridClock initialized: now 1786056301950271 us; error 30 us; skew 500 ppm
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:01.952872 3420 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:45:01.953284 3420 webserver.cc:460] Webserver started at http://127.19.74.195:39539/ using document root <none> and password file <none>
I20260806 22:45:01.953511 3420 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:01.953578 3420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:01.953733 3420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:01.954782 3420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "bf0bfc169191452b94a504aec98cf82c"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "a57195009afa283fcc523e6fffb48d0d"
server_key_iv: "4f31c2d5ca967c1a422b419f00d58b9d"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.955124 3420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "bf0bfc169191452b94a504aec98cf82c"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "a57195009afa283fcc523e6fffb48d0d"
server_key_iv: "4f31c2d5ca967c1a422b419f00d58b9d"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.956490 3420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:45:01.957250 3437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.957422 3420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:45:01.957531 3420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "bf0bfc169191452b94a504aec98cf82c"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "a57195009afa283fcc523e6fffb48d0d"
server_key_iv: "4f31c2d5ca967c1a422b419f00d58b9d"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.957603 3420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:01.970535 3420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:01.972843 3420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:01.973008 3420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:01.973237 3420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:01.973557 3420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:01.973628 3420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.973664 3420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:01.973691 3420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.979558 3420 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:36083
I20260806 22:45:01.979604 3554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:36083 every 8 connection(s)
I20260806 22:45:01.979938 3420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:01.982077 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 3420
I20260806 22:45:01.982190 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:01.982398 19755 external_mini_cluster.cc:1428] Setting key 8f5bbf2ab0d00215e6781445d59ea727
Aug 06 22:45:01 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056301, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:45:01.989603 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.980689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:55931 (local address 127.19.74.254:38903)
0806 22:45:01.980768 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.980771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.981008 (+ 237us) server_negotiation.cc:407] Connection header received
0806 22:45:01.981471 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.981472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.981483 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.981537 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.982829 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.983579 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.984559 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.984743 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.984795 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:01.987197 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.987198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.987198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:01.987205 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.988423 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:01.988800 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:01.988801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:01.988802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:01.988855 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:01.989067 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:01.989068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:01.989068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:01.989184 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.989235 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.989472 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.989536 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:45:01.989888 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.980688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:55931)
0806 22:45:01.980883 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.980918 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.981379 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.981585 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.981589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.982354 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.982685 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:01.982694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.983676 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.983677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:01.984485 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:01.984488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.984892 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.984893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:01.985012 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:01.985470 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:01.985479 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.987103 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.988498 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:01.988504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:01.988506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:01.988734 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:01.988914 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:01.988915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:01.988915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:01.989015 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:01.989240 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.989245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.989361 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.989442 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.989564 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260806 22:45:01.990401 3555 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38903
I20260806 22:45:01.990528 3555 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:01.990757 3555 heartbeater.cc:511] Master 127.19.74.254:38903 requested a full tablet report, sending...
I20260806 22:45:01.991220 3079 ts_manager.cc:194] Registered new tserver with Master: bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
I20260806 22:45:01.991659 3079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:55931
I20260806 22:45:01.994221 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:45:02.001302 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.995522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.19.74.254:38903)
0806 22:45:01.995586 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.995589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.995682 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:45:01.995755 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.995756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.995766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.995834 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.996371 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.996825 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.997520 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:01.997727 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:01.997829 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:01.999034 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.999035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.999036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:01.999042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:02.000102 (+ 1060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:02.000474 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:02.000475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:02.000475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:02.000536 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:02.000747 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:02.000748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:02.000748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:02.000894 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:02.000998 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.001125 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.001201 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:45:02.002975 3079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:02.004576 3079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:02.011555 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.007028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:36083 (local address 127.0.0.1:47994)
0806 22:45:02.007405 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.007416 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:02.007469 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:02.007823 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:02.007824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:02.007827 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:02.007873 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.007874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.009041 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.009042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.010201 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.010205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.011279 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.011281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.011356 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.011361 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:02.011417 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:45:02.011462 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":1,"thread_start_us":38,"threads_started":1}
I20260806 22:45:02.011570 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.007236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47994 (local address 127.19.74.195:36083)
0806 22:45:02.007587 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.007611 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.007621 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:02.007658 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.007661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.007676 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.007744 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:02.007937 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.008966 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.010690 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.011203 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.011276 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.011334 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.011444 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.011487 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":92,"threads_started":1}
I20260806 22:45:02.011816 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.007043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.19.74.193:45911)
0806 22:45:02.007210 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.007216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.007228 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:02.007275 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.007277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.007294 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.007370 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:02.007620 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.008715 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.010314 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.011087 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.011163 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.011218 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.011515 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.011566 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260806 22:45:02.012115 3565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.006837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:45911 (local address 127.0.0.1:58424)
0806 22:45:02.007066 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.007110 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:02.007181 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:02.007459 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:02.007469 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:02.007471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:02.007556 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.007557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.008823 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.008823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.010205 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.010206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.011180 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.011182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.011273 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.011290 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:45:02.011360 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:45:02.011443 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":43,"thread_start_us":56,"threads_started":1}
I20260806 22:45:02.012341 3485 tablet_service.cc:1467] Processing CreateTablet for tablet 7c48dd42f1f24d91a61e49a441878baf (DEFAULT_TABLE table=test-table [id=159dfe1cfda94c0eb58848aafbe63779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:02.012723 3485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c48dd42f1f24d91a61e49a441878baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:02.012845 3207 tablet_service.cc:1467] Processing CreateTablet for tablet 7c48dd42f1f24d91a61e49a441878baf (DEFAULT_TABLE table=test-table [id=159dfe1cfda94c0eb58848aafbe63779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:02.013201 3207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c48dd42f1f24d91a61e49a441878baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:02.015656 3570 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Bootstrap starting.
I20260806 22:45:02.016022 3571 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Bootstrap starting.
I20260806 22:45:02.016369 3570 tablet_bootstrap.cc:654] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:02.016485 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.006837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:39563 (local address 127.0.0.1:53950)
0806 22:45:02.007279 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.007289 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:02.007340 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:02.007793 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:02.007793 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:02.007796 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:02.007874 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.007875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.010117 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.010118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.011284 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.011286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.016193 (+ 4907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.016194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.016268 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.016274 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:45:02.016338 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:45:02.016406 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":5,"thread_start_us":56,"threads_started":1}
I20260806 22:45:02.016721 3571 tablet_bootstrap.cc:654] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:02.016775 3566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.006943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53950 (local address 127.19.74.194:39563)
0806 22:45:02.007263 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.007269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.007336 (+ 67us) server_negotiation.cc:407] Connection header received
0806 22:45:02.007409 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.007412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.007432 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.007707 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:02.009069 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.010021 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.011373 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.012417 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.016083 (+ 3666us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.016150 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.016376 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.016513 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":111,"threads_started":1}
I20260806 22:45:02.017076 3570 log.cc:824] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:02.017270 3571 log.cc:824] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:02.017531 3346 tablet_service.cc:1467] Processing CreateTablet for tablet 7c48dd42f1f24d91a61e49a441878baf (DEFAULT_TABLE table=test-table [id=159dfe1cfda94c0eb58848aafbe63779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:02.017891 3346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c48dd42f1f24d91a61e49a441878baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:02.017961 3570 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: No bootstrap required, opened a new log
I20260806 22:45:02.018028 3570 ts_tablet_manager.cc:1397] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:02.018046 3571 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: No bootstrap required, opened a new log
I20260806 22:45:02.018174 3571 ts_tablet_manager.cc:1397] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:02.019721 3571 raft_consensus.cc:348] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.019881 3571 raft_consensus.cc:374] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:02.019927 3571 raft_consensus.cc:729] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41dc974f6da145f3a63842c794d027e0, State: Initialized, Role: FOLLOWER
I20260806 22:45:02.020084 3571 consensus_queue.cc:260] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.020009 3570 raft_consensus.cc:348] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.020185 3570 raft_consensus.cc:374] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:02.020251 3570 raft_consensus.cc:729] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0bfc169191452b94a504aec98cf82c, State: Initialized, Role: FOLLOWER
I20260806 22:45:02.020357 3570 consensus_queue.cc:260] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.020388 3571 ts_tablet_manager.cc:1428] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:02.020426 3278 heartbeater.cc:503] Master 127.19.74.254:38903 was elected leader, sending a full tablet report...
I20260806 22:45:02.020646 3555 heartbeater.cc:503] Master 127.19.74.254:38903 was elected leader, sending a full tablet report...
I20260806 22:45:02.020628 3570 ts_tablet_manager.cc:1428] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:02.021385 3576 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Bootstrap starting.
I20260806 22:45:02.022006 3576 tablet_bootstrap.cc:654] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:02.022455 3576 log.cc:824] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:02.023165 3576 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: No bootstrap required, opened a new log
I20260806 22:45:02.023244 3576 ts_tablet_manager.cc:1397] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:02.024493 3576 raft_consensus.cc:348] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.024642 3576 raft_consensus.cc:374] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:02.024688 3576 raft_consensus.cc:729] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af444ef9234e445a8e4479a4d7f61da2, State: Initialized, Role: FOLLOWER
I20260806 22:45:02.024794 3576 consensus_queue.cc:260] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.025024 3576 ts_tablet_manager.cc:1428] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:02.025061 3415 heartbeater.cc:503] Master 127.19.74.254:38903 was elected leader, sending a full tablet report...
W20260806 22:45:02.074121 3416 tablet.cc:2369] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:45:02.176433 3279 tablet.cc:2369] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:45:02.230540 3556 tablet.cc:2369] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:02.348845 3575 raft_consensus.cc:482] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:02.348969 3575 raft_consensus.cc:504] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.349417 3575 leader_election.cc:288] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), 41dc974f6da145f3a63842c794d027e0 (127.19.74.193:45911)
I20260806 22:45:02.352345 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.349705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:39013 (local address 127.19.74.193:45911)
0806 22:45:02.349747 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.349749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.349867 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:45:02.349895 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.349896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.349908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.349963 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:02.350124 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.350672 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.351612 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.351973 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.352058 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.352066 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.352201 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.352275 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:45:02.352350 3579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.349636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:45911 (local address 127.19.74.195:39013)
0806 22:45:02.349784 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.349792 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:02.349855 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:02.349972 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:02.349973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:02.349974 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:02.350056 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.350058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.350747 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.350748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.351506 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.351508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.352056 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.352057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.352127 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.352130 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:02.352172 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:45:02.352201 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260806 22:45:02.352363 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.349616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:39563 (local address 127.19.74.195:37573)
0806 22:45:02.349673 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.349684 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:02.349727 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:02.349885 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:02.349886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:02.349887 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:02.349962 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.349964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.350711 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.350712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.351504 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.351506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.352063 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.352064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.352171 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.352175 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:02.352260 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:45:02.352295 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":32}
I20260806 22:45:02.352406 3566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.349667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:37573 (local address 127.19.74.194:39563)
0806 22:45:02.349711 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.349713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.349723 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:02.349796 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.349797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.349811 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.349863 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:02.350035 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.350580 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.351603 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.351974 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.352053 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.352062 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.352290 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.352331 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:45:02.352787 3228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c48dd42f1f24d91a61e49a441878baf" candidate_uuid: "bf0bfc169191452b94a504aec98cf82c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41dc974f6da145f3a63842c794d027e0" is_pre_election: true
I20260806 22:45:02.352805 3366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c48dd42f1f24d91a61e49a441878baf" candidate_uuid: "bf0bfc169191452b94a504aec98cf82c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af444ef9234e445a8e4479a4d7f61da2" is_pre_election: true
I20260806 22:45:02.352921 3228 raft_consensus.cc:2454] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0bfc169191452b94a504aec98cf82c in term 0.
I20260806 22:45:02.352958 3366 raft_consensus.cc:2454] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0bfc169191452b94a504aec98cf82c in term 0.
I20260806 22:45:02.353200 3440 leader_election.cc:302] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41dc974f6da145f3a63842c794d027e0, bf0bfc169191452b94a504aec98cf82c; no voters:
I20260806 22:45:02.353335 3575 raft_consensus.cc:2781] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:02.353392 3575 raft_consensus.cc:482] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:02.353420 3575 raft_consensus.cc:3037] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:02.354107 3575 raft_consensus.cc:504] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.354249 3575 leader_election.cc:288] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [CANDIDATE]: Term 1 election: Requested vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), 41dc974f6da145f3a63842c794d027e0 (127.19.74.193:45911)
I20260806 22:45:02.354403 3366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c48dd42f1f24d91a61e49a441878baf" candidate_uuid: "bf0bfc169191452b94a504aec98cf82c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af444ef9234e445a8e4479a4d7f61da2"
I20260806 22:45:02.354430 3228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c48dd42f1f24d91a61e49a441878baf" candidate_uuid: "bf0bfc169191452b94a504aec98cf82c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41dc974f6da145f3a63842c794d027e0"
I20260806 22:45:02.354487 3366 raft_consensus.cc:3037] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:02.354497 3228 raft_consensus.cc:3037] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:02.355386 3366 raft_consensus.cc:2454] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0bfc169191452b94a504aec98cf82c in term 1.
I20260806 22:45:02.355386 3228 raft_consensus.cc:2454] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0bfc169191452b94a504aec98cf82c in term 1.
I20260806 22:45:02.355564 3438 leader_election.cc:302] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af444ef9234e445a8e4479a4d7f61da2, bf0bfc169191452b94a504aec98cf82c; no voters:
I20260806 22:45:02.355661 3575 raft_consensus.cc:2781] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:02.355825 3575 raft_consensus.cc:686] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 1 LEADER]: Becoming Leader. State: Replica: bf0bfc169191452b94a504aec98cf82c, State: Running, Role: LEADER
I20260806 22:45:02.355932 3575 consensus_queue.cc:237] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:02.356838 3079 catalog_manager.cc:5495] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0bfc169191452b94a504aec98cf82c (127.19.74.195). New cstate: current_term: 1 leader_uuid: "bf0bfc169191452b94a504aec98cf82c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } health_report { overall_health: HEALTHY } } }
I20260806 22:45:02.362360 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.359956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.19.74.195:36083)
0806 22:45:02.359992 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.359995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.360043 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:45:02.360135 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.360136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.360152 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.360228 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:02.360361 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.360817 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.361655 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.361837 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.361903 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.362057 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:02.362155 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:02.362242 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.362261 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.362277 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:45:02.364053 3366 raft_consensus.cc:1261] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 1 FOLLOWER]: Refusing update from remote peer bf0bfc169191452b94a504aec98cf82c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:02.364053 3228 raft_consensus.cc:1261] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Refusing update from remote peer bf0bfc169191452b94a504aec98cf82c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:02.364513 3575 consensus_queue.cc:1035] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [LEADER]: Connected to new peer: Peer: permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:02.364668 3581 consensus_queue.cc:1035] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [LEADER]: Connected to new peer: Peer: permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:02.366991 3584 mvcc.cc:205] Tried to move back new op lower bound from 7315686614480494592 to 7315686614450442240. Current Snapshot: MvccSnapshot[applied={T|T < 7315686614480494592}]
I20260806 22:45:02.367172 3586 mvcc.cc:205] Tried to move back new op lower bound from 7315686614480494592 to 7315686614450442240. Current Snapshot: MvccSnapshot[applied={T|T < 7315686614480494592}]
I20260806 22:45:02.367278 3585 mvcc.cc:205] Tried to move back new op lower bound from 7315686614480494592 to 7315686614450442240. Current Snapshot: MvccSnapshot[applied={T|T < 7315686614480494592}]
I20260806 22:45:02.369791 3079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:02.369874 3079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:02.370672 3079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Sending DeleteTablet for 3 replicas of tablet 7c48dd42f1f24d91a61e49a441878baf
I20260806 22:45:02.370971 3346 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c48dd42f1f24d91a61e49a441878baf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:02 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:53950
I20260806 22:45:02.371013 3207 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c48dd42f1f24d91a61e49a441878baf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:02 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:58424
I20260806 22:45:02.371111 3485 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c48dd42f1f24d91a61e49a441878baf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:02 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:47994
I20260806 22:45:02.371227 3595 tablet_replica.cc:330] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: stopping tablet replica
I20260806 22:45:02.371234 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 3144
I20260806 22:45:02.373718 3594 tablet_replica.cc:330] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: stopping tablet replica
I20260806 22:45:02.373842 3594 raft_consensus.cc:2229] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:02.374006 3594 raft_consensus.cc:2258] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:02.374233 3597 tablet_replica.cc:330] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: stopping tablet replica
I20260806 22:45:02.374503 3597 raft_consensus.cc:2229] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:02.374548 3594 ts_tablet_manager.cc:1905] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:02.374761 3597 raft_consensus.cc:2258] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:02.376199 3594 ts_tablet_manager.cc:1918] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:02.376276 3594 log.cc:1196] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/7c48dd42f1f24d91a61e49a441878baf
I20260806 22:45:02.376566 3594 ts_tablet_manager.cc:1939] T 7c48dd42f1f24d91a61e49a441878baf P af444ef9234e445a8e4479a4d7f61da2: Deleting consensus metadata
I20260806 22:45:02.376991 3062 catalog_manager.cc:4841] TS af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563): tablet 7c48dd42f1f24d91a61e49a441878baf (table test-table [id=159dfe1cfda94c0eb58848aafbe63779]) successfully deleted
I20260806 22:45:02.380072 3597 ts_tablet_manager.cc:1905] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:02.382084 3597 ts_tablet_manager.cc:1918] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:02.382177 3597 log.cc:1196] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/7c48dd42f1f24d91a61e49a441878baf
I20260806 22:45:02.382511 3597 ts_tablet_manager.cc:1939] T 7c48dd42f1f24d91a61e49a441878baf P bf0bfc169191452b94a504aec98cf82c: Deleting consensus metadata
I20260806 22:45:02.382992 3062 catalog_manager.cc:4841] TS bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083): tablet 7c48dd42f1f24d91a61e49a441878baf (table test-table [id=159dfe1cfda94c0eb58848aafbe63779]) successfully deleted
W20260806 22:45:02.384481 3064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:45911 (error 108)
I20260806 22:45:02.385030 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.384889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:45911 (local address 127.0.0.1:58436)
0806 22:45:02.384949 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.384993 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:45911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:45:02.385131 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 3283
W20260806 22:45:02.387204 3064 catalog_manager.cc:4584] TS 41dc974f6da145f3a63842c794d027e0 (127.19.74.193:45911): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c48dd42f1f24d91a61e49a441878baf: Network error: Client connection negotiation failed: client connection to 127.19.74.193:45911: connect: Connection refused (error 111)
I20260806 22:45:02.395596 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 3420
I20260806 22:45:02.401687 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 3046
I20260806 22:45:22.409869 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38903
--webserver_interface=127.19.74.254
--webserver_port=36077
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:22.492790 3599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:22.492995 3599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:22.493017 3599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:22.494131 3599 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:45:22.494164 3599 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:45:22.494179 3599 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:45:22.494189 3599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:22.494202 3599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:22.494215 3599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:22.494226 3599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:22.494243 3599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:22.495973 3599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=36077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:22.496199 3599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:22.496384 3599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:22.506013 3607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.506090 3599 server_base.cc:1034] running on GCE node
W20260806 22:45:22.506323 3605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.506323 3609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.506641 3599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:22.506990 3599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:22.508174 3599 hybrid_clock.cc:648] HybridClock initialized: now 1786056322508168 us; error 36 us; skew 500 ppm
Aug 06 22:45:22 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:22.516603 3599 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:45:22.517081 3599 webserver.cc:460] Webserver started at http://127.19.74.254:36077/ using document root <none> and password file <none>
I20260806 22:45:22.517334 3599 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:22.517427 3599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:22.518900 3599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:22.519657 3616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.519836 3599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:45:22.519953 3599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "325b9f4c914d4c24af9a021d3bef614d"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "b136bf4b406649d02cc4ba91c438532a"
server_key_iv: "38961aef9755ca7d5055520274bb47e6"
server_key_version: "encryptionkey@0"
I20260806 22:45:22.520388 3599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:22.551919 3599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:22.553859 3599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:22.554033 3599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:22.558144 3599 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:38903
I20260806 22:45:22.558200 3671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:38903 every 8 connection(s)
I20260806 22:45:22.558605 3599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:22.560151 3672 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:45:22.560844 3672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Bootstrap starting.
I20260806 22:45:22.566345 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 3599
I20260806 22:45:22.566735 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:45911
--local_ip_for_outbound_sockets=127.19.74.193
--tserver_master_addrs=127.19.74.254:38903
--webserver_port=38925
--webserver_interface=127.19.74.193
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:45:22.569965 3672 log.cc:824] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:22.572062 3672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:45:22.572381 3672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Bootstrap complete.
I20260806 22:45:22.574786 3672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:22.575040 3672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325b9f4c914d4c24af9a021d3bef614d, State: Initialized, Role: FOLLOWER
I20260806 22:45:22.575389 3672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:22.575532 3672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:22.575606 3672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:22.575663 3672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:45:22.576807 3672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:22.576936 3672 leader_election.cc:302] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 325b9f4c914d4c24af9a021d3bef614d; no voters:
I20260806 22:45:22.577198 3672 leader_election.cc:288] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:45:22.577239 3676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:45:22.577368 3676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 2 LEADER]: Becoming Leader. State: Replica: 325b9f4c914d4c24af9a021d3bef614d, State: Running, Role: LEADER
I20260806 22:45:22.577497 3672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:22.577561 3676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:22.577962 3677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "325b9f4c914d4c24af9a021d3bef614d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } } }
I20260806 22:45:22.577983 3678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 325b9f4c914d4c24af9a021d3bef614d. Latest consensus state: current_term: 2 leader_uuid: "325b9f4c914d4c24af9a021d3bef614d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } } }
I20260806 22:45:22.578059 3677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:22.578080 3678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:22.578296 3684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:22.579267 3684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:22.579440 3684 catalog_manager.cc:1259] Loaded cluster ID: e582665784b64360b1f322694323a5a2
I20260806 22:45:22.579690 3684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:22.581015 3684 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:22.581252 3684 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Loaded TSK: 0
I20260806 22:45:22.601862 3684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Generated new TSK 1
I20260806 22:45:22.601969 3684 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:45:22.656566 3674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:22.656795 3674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:22.656833 3674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:22.657984 3674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:22.658049 3674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:22.658102 3674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:45:22.659504 3674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:45911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=38925
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:22.659766 3674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:22.659977 3674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:22.664971 3703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.664984 3700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.664984 3701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.665537 3674 server_base.cc:1034] running on GCE node
I20260806 22:45:22.665759 3674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:22.666018 3674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:22.667167 3674 hybrid_clock.cc:648] HybridClock initialized: now 1786056322667152 us; error 40 us; skew 500 ppm
Aug 06 22:45:22 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:22.669713 3674 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:45:22.670094 3674 webserver.cc:460] Webserver started at http://127.19.74.193:38925/ using document root <none> and password file <none>
I20260806 22:45:22.670303 3674 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:22.670380 3674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:22.671799 3674 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:22.672592 3710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.672760 3674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:22.672859 3674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "41dc974f6da145f3a63842c794d027e0"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "aa7647354dafd39067c2771240e0e4e2"
server_key_iv: "b05d34fe6f97d6cb84191bc804997b59"
server_key_version: "encryptionkey@0"
I20260806 22:45:22.673208 3674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:22.696128 3674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:22.698675 3674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:22.698870 3674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:22.699115 3674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:22.699579 3717 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:45:22.701712 3674 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:45:22.701774 3674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:45:22.701821 3674 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:45:22.702602 3674 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:45:22.702693 3674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:22.702752 3717 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Bootstrap starting.
I20260806 22:45:22.709488 3674 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:45911
I20260806 22:45:22.709609 3828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:45911 every 8 connection(s)
I20260806 22:45:22.709939 3674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:22.712890 3717 log.cc:824] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:22.713263 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 3674
I20260806 22:45:22.713574 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:39563
--local_ip_for_outbound_sockets=127.19.74.194
--tserver_master_addrs=127.19.74.254:38903
--webserver_port=40823
--webserver_interface=127.19.74.194
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:45:22.715757 3717 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:45:22.716100 3717 tablet_bootstrap.cc:492] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Bootstrap complete.
I20260806 22:45:22.717248 3717 ts_tablet_manager.cc:1397] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.010s
Aug 06 22:45:22 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:45:22.718199 3717 raft_consensus.cc:348] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:22.718461 3717 raft_consensus.cc:729] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41dc974f6da145f3a63842c794d027e0, State: Initialized, Role: FOLLOWER
I20260806 22:45:22.718642 3717 consensus_queue.cc:260] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } }
I20260806 22:45:22.719035 3717 ts_tablet_manager.cc:1428] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:22.721907 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.710752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:48923 (local address 127.19.74.254:38903)
0806 22:45:22.710942 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.710950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.711036 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:45:22.711853 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.711858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.711900 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.712001 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.713008 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.714206 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.715308 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.715917 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.716026 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.718712 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:22.718716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:22.718716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:22.718727 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:22.720241 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.720733 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.720735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.720736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.720814 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.721087 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.721088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.721088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.721290 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:22.721347 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.721691 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.721782 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
I20260806 22:45:22.722034 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.710654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:48923)
0806 22:45:22.710859 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.710906 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.711658 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.712088 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.712095 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.712562 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.712841 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.712847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.714334 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.714336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.715185 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.715187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.716029 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.716031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.716186 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.716794 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.716804 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.718588 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:22.720321 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:22.720329 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:22.720331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:22.720641 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:22.720877 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:22.720878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:22.720878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:22.721020 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:22.721356 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:22.721364 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:22.721533 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 22:45:22.721639 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:45:22.721811 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260806 22:45:22.722820 3829 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38903
I20260806 22:45:22.722927 3829 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:22.723155 3829 heartbeater.cc:511] Master 127.19.74.254:38903 requested a full tablet report, sending...
I20260806 22:45:22.723970 3634 ts_manager.cc:194] Registered new tserver with Master: 41dc974f6da145f3a63842c794d027e0 (127.19.74.193:45911)
I20260806 22:45:22.725198 3634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:48923
I20260806 22:45:22.726137 3829 heartbeater.cc:503] Master 127.19.74.254:38903 was elected leader, sending a full tablet report...
I20260806 22:45:22.729414 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.725627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:45911 (local address 127.0.0.1:58754)
0806 22:45:22.725796 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.725829 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.725887 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.726193 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.726196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.726198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:22.726253 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.726255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.727263 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.727264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.728207 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.728210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.729087 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.729088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.729186 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.729202 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:45:22.729279 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:45:22.729338 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260806 22:45:22.729457 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.725873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.19.74.193:45911)
0806 22:45:22.726015 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.726019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.726028 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:22.726061 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.726063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.726077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.726173 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:22.726329 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.727168 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.728281 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.728931 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.729012 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.729061 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.729310 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.729378 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260806 22:45:22.729925 3760 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c48dd42f1f24d91a61e49a441878baf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:02 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:58754
I20260806 22:45:22.730448 3838 tablet_replica.cc:330] stopping tablet replica
I20260806 22:45:22.730541 3838 raft_consensus.cc:2229] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:22.730599 3838 raft_consensus.cc:2258] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:22.730918 3838 ts_tablet_manager.cc:1905] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:22.733398 3838 ts_tablet_manager.cc:1918] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:22.733472 3838 log.cc:1196] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal/wals/7c48dd42f1f24d91a61e49a441878baf
I20260806 22:45:22.733695 3838 ts_tablet_manager.cc:1939] T 7c48dd42f1f24d91a61e49a441878baf P 41dc974f6da145f3a63842c794d027e0: Deleting consensus metadata
I20260806 22:45:22.734184 3619 catalog_manager.cc:4841] TS 41dc974f6da145f3a63842c794d027e0 (127.19.74.193:45911): tablet 7c48dd42f1f24d91a61e49a441878baf (table test-table [id=159dfe1cfda94c0eb58848aafbe63779]) successfully deleted
W20260806 22:45:22.802276 3834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:22.802475 3834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:22.802498 3834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:22.803634 3834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:22.803678 3834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:22.803702 3834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:45:22.805078 3834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:39563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=40823
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:22.805327 3834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:22.805546 3834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:22.810424 3844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.810380 3846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.810503 3834 server_base.cc:1034] running on GCE node
W20260806 22:45:22.810447 3843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.810797 3834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:22.811043 3834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:22.812213 3834 hybrid_clock.cc:648] HybridClock initialized: now 1786056322812202 us; error 28 us; skew 500 ppm
Aug 06 22:45:22 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:22.814563 3834 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:45:22.815023 3834 webserver.cc:460] Webserver started at http://127.19.74.194:40823/ using document root <none> and password file <none>
I20260806 22:45:22.815263 3834 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:22.815346 3834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:22.816691 3834 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:22.817337 3853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.817529 3834 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:45:22.817632 3834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "af444ef9234e445a8e4479a4d7f61da2"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "1ab88d4df3db650a2c848c9d9c7b65cb"
server_key_iv: "7906755dce9569eaedf3805a55ea907d"
server_key_version: "encryptionkey@0"
I20260806 22:45:22.817961 3834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:22.826972 3834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:22.828689 3834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:22.828845 3834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:22.829056 3834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:22.829380 3834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:22.829433 3834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.829474 3834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:22.829504 3834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.835594 3834 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:39563
I20260806 22:45:22.835655 3973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:39563 every 8 connection(s)
I20260806 22:45:22.835992 3834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:22.840927 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 3834
I20260806 22:45:22.841202 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:36083
--local_ip_for_outbound_sockets=127.19.74.195
--tserver_master_addrs=127.19.74.254:38903
--webserver_port=39539
--webserver_interface=127.19.74.195
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:45:22 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:45:22.845100 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.836705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:37989 (local address 127.19.74.254:38903)
0806 22:45:22.836778 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.836781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.836979 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:45:22.837489 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.837490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.837503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.837561 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.838419 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.838980 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.839882 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.840074 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.840123 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.842262 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:22.842282 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:22.842282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:22.842292 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:22.843691 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.844099 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.844101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.844101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.844160 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.844439 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.844439 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.844440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.844562 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:22.844633 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.844896 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.845008 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:45:22.845307 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.836655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:37989)
0806 22:45:22.836830 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.836871 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.837400 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.837621 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.837624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.838160 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.838341 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.838345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.839055 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.839055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.839812 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.839815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.840139 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.840140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.840235 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.840737 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.840745 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.842163 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:22.843792 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:22.843799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:22.843801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:22.844015 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:22.844230 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:22.844231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:22.844231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:22.844348 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:22.844635 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:22.844643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:22.844773 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 22:45:22.844874 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:45:22.845057 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260806 22:45:22.845775 3974 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38903
I20260806 22:45:22.845867 3974 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:22.846062 3974 heartbeater.cc:511] Master 127.19.74.254:38903 requested a full tablet report, sending...
I20260806 22:45:22.846494 3634 ts_manager.cc:194] Registered new tserver with Master: af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563)
I20260806 22:45:22.847052 3634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:37989
W20260806 22:45:22.924870 3978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:22.925064 3978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:22.925086 3978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:22.926137 3978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:22.926172 3978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:22.926195 3978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:45:22.927640 3978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:36083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=39539
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:22.927825 3978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:22.927991 3978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:22.932806 3986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.932804 3984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.932875 3983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.933157 3978 server_base.cc:1034] running on GCE node
I20260806 22:45:22.933350 3978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:22.933599 3978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:22.934746 3978 hybrid_clock.cc:648] HybridClock initialized: now 1786056322934734 us; error 22 us; skew 500 ppm
Aug 06 22:45:22 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:22.937143 3978 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:45:22.937659 3978 webserver.cc:460] Webserver started at http://127.19.74.195:39539/ using document root <none> and password file <none>
I20260806 22:45:22.937846 3978 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:22.937894 3978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:22.939186 3978 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:22.939687 3993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.939864 3978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.939962 3978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "bf0bfc169191452b94a504aec98cf82c"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "a57195009afa283fcc523e6fffb48d0d"
server_key_iv: "4f31c2d5ca967c1a422b419f00d58b9d"
server_key_version: "encryptionkey@0"
I20260806 22:45:22.940310 3978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:22.952566 3978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:22.954270 3978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:22.954442 3978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:22.954707 3978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:22.955044 3978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:22.955097 3978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.955130 3978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:22.955159 3978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.960822 3978 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:36083
I20260806 22:45:22.960873 4114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:36083 every 8 connection(s)
I20260806 22:45:22.961194 3978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:45:22 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:45:22.967120 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 3978
I20260806 22:45:22.970121 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.961842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:60491 (local address 127.19.74.254:38903)
0806 22:45:22.961903 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.961905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.962097 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:45:22.962650 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.962651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.962665 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.962729 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.963655 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.964132 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.965067 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.965270 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.965346 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.967393 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:22.967394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:22.967394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:22.967403 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:22.968850 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.969217 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.969218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.969219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.969285 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.969551 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.969552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.969552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.969647 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:22.969695 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.970000 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.970057 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:45:22.970328 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.961841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:60491)
0806 22:45:22.961984 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.962016 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.962510 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.962806 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.962809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.963379 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.963553 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.963557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.964225 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.964226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.964977 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.964979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.965350 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.965352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.965477 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.965888 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.965896 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.967291 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:22.968937 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:22.968942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:22.968944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:22.969148 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:22.969355 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:22.969356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:22.969357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:22.969485 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:22.969717 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:22.969723 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:22.969864 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 22:45:22.969980 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:45:22.970122 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260806 22:45:22.970858 4115 heartbeater.cc:347] Connected to a master server at 127.19.74.254:38903
I20260806 22:45:22.970983 4115 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:22.971199 4115 heartbeater.cc:511] Master 127.19.74.254:38903 requested a full tablet report, sending...
I20260806 22:45:22.971654 3634 ts_manager.cc:194] Registered new tserver with Master: bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
I20260806 22:45:22.972122 3634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:60491
I20260806 22:45:22.974354 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.967498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42680 (local address 127.19.74.254:38903)
0806 22:45:22.967671 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.967672 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.967696 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:45:22.967787 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.967788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.967799 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.967858 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.968541 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.969197 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.970230 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.970509 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.970586 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.971989 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:22.971990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:22.971991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:22.971996 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:22.973383 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.973705 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.973706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.973706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.973755 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.973899 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.973900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.973900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.973978 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:22.974115 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.974151 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.974223 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260806 22:45:22.975078 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:45:22.977607 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.975411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.19.74.254:38903)
0806 22:45:22.975447 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.975449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.975528 (+ 79us) server_negotiation.cc:407] Connection header received
0806 22:45:22.975587 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.975587 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.975615 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.975671 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:22.975773 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.976201 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.976942 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.977140 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.977190 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.977326 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:22.977409 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:22.977464 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.977494 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.977511 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:45:22.977869 3634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:22.979598 3634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:22.981626 3760 tablet_service.cc:1467] Processing CreateTablet for tablet f2bd57b5d0a84ae9851252bb24848c26 (DEFAULT_TABLE table=test-table [id=0e3d8368223849cc97bc1bd025f0f581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:22.981832 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2bd57b5d0a84ae9851252bb24848c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:22.983461 3717 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Bootstrap starting.
I20260806 22:45:22.983785 3717 tablet_bootstrap.cc:654] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:22.984378 3717 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: No bootstrap required, opened a new log
I20260806 22:45:22.984432 3717 ts_tablet_manager.cc:1397] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:22.984539 3717 raft_consensus.cc:348] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:22.984615 3717 raft_consensus.cc:374] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:22.984661 3717 raft_consensus.cc:729] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41dc974f6da145f3a63842c794d027e0, State: Initialized, Role: FOLLOWER
I20260806 22:45:22.984704 3717 consensus_queue.cc:260] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:22.984804 3717 ts_tablet_manager.cc:1428] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.985402 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.981422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:36083 (local address 127.0.0.1:41160)
0806 22:45:22.981576 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.981586 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.981648 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.982019 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.982020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.982023 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:22.982059 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.982061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.983378 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.983379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.984278 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.984281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.985184 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.985185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.985250 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.985254 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:22.985289 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 22:45:22.985330 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260806 22:45:22.985579 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.981316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:39563 (local address 127.0.0.1:49016)
0806 22:45:22.981365 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.981375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.981411 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.981806 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.981807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.981809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:22.981845 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.981846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.983175 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.983176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.984262 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.984266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.985368 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.985369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.985427 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.985430 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:22.985486 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:45:22.985525 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:45:22.985744 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.981629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.19.74.195:36083)
0806 22:45:22.981865 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.981869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.981878 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:22.981911 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.981913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.981926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.982008 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:22.982120 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.983298 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.984381 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.985086 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.985508 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.985571 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.985608 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.985651 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":86,"threads_started":1}
I20260806 22:45:22.985741 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.981422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49016 (local address 127.19.74.194:39563)
0806 22:45:22.981652 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.981657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.981667 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:22.981710 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.981712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.981727 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.981805 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:22.981909 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.983076 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.984461 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.985243 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.985310 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.985369 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.985594 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.985642 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":112,"threads_started":1}
I20260806 22:45:22.986510 4045 tablet_service.cc:1467] Processing CreateTablet for tablet f2bd57b5d0a84ae9851252bb24848c26 (DEFAULT_TABLE table=test-table [id=0e3d8368223849cc97bc1bd025f0f581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:22.986510 3904 tablet_service.cc:1467] Processing CreateTablet for tablet f2bd57b5d0a84ae9851252bb24848c26 (DEFAULT_TABLE table=test-table [id=0e3d8368223849cc97bc1bd025f0f581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:22.986886 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2bd57b5d0a84ae9851252bb24848c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:22.986886 4045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2bd57b5d0a84ae9851252bb24848c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:22.990317 4125 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Bootstrap starting.
I20260806 22:45:22.990483 4126 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: Bootstrap starting.
I20260806 22:45:22.991039 4125 tablet_bootstrap.cc:654] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:22.991111 4126 tablet_bootstrap.cc:654] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:22.991708 4125 log.cc:824] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:22.991708 4126 log.cc:824] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:22.992489 4126 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: No bootstrap required, opened a new log
I20260806 22:45:22.992502 4125 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: No bootstrap required, opened a new log
I20260806 22:45:22.992576 4126 ts_tablet_manager.cc:1397] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:22.992568 4125 ts_tablet_manager.cc:1397] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:22.994144 4126 raft_consensus.cc:348] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:22.994195 4125 raft_consensus.cc:348] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:22.994289 4126 raft_consensus.cc:374] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:22.994349 4125 raft_consensus.cc:374] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:22.994354 4126 raft_consensus.cc:729] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0bfc169191452b94a504aec98cf82c, State: Initialized, Role: FOLLOWER
I20260806 22:45:22.994395 4125 raft_consensus.cc:729] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af444ef9234e445a8e4479a4d7f61da2, State: Initialized, Role: FOLLOWER
I20260806 22:45:22.994489 4126 consensus_queue.cc:260] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:22.994500 4125 consensus_queue.cc:260] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:22.994791 4115 heartbeater.cc:503] Master 127.19.74.254:38903 was elected leader, sending a full tablet report...
I20260806 22:45:22.994813 4125 ts_tablet_manager.cc:1428] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:45:22.994799 4126 ts_tablet_manager.cc:1428] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:45:22.995049 3974 heartbeater.cc:503] Master 127.19.74.254:38903 was elected leader, sending a full tablet report...
W20260806 22:45:23.086771 3975 tablet.cc:2369] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:23.130677 3835 raft_consensus.cc:482] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:23.130806 3835 raft_consensus.cc:504] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:23.131098 3835 leader_election.cc:288] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
I20260806 22:45:23.134140 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.131528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:60675 (local address 127.19.74.195:36083)
0806 22:45:23.131575 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:23.131577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:23.131657 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:45:23.131732 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:23.131734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:23.131743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:23.131792 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:23.131933 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.132426 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.133402 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.133790 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.133875 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.133885 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:23.134035 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 22:45:23.134073 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:45:23.134138 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.131404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:50395 (local address 127.19.74.194:39563)
0806 22:45:23.131459 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:23.131462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:23.131472 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:23.131503 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:23.131504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:23.131514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:23.131572 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:23.132007 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.132490 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.133399 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.133779 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.133863 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.133872 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:23.134017 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:45:23.134056 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:45:23.134156 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.131275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:39563 (local address 127.19.74.193:50395)
0806 22:45:23.131339 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:23.131353 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:23.131423 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:23.131577 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:23.131579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:23.131580 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:23.131919 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.131921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.132575 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.132576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.133305 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.133307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.133860 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.133861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.133931 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.133934 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:23.134007 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:45:23.134051 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":1}
I20260806 22:45:23.134406 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.131411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:36083 (local address 127.19.74.193:60675)
0806 22:45:23.131579 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:23.131589 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:23.131670 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:23.131795 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:23.131795 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:23.131797 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:23.131866 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.131868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.132513 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.132514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.133306 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.133308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.133894 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.133895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.133962 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.133965 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:23.134030 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:45:23.134068 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":109,"threads_started":1}
I20260806 22:45:23.134572 3924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2bd57b5d0a84ae9851252bb24848c26" candidate_uuid: "41dc974f6da145f3a63842c794d027e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af444ef9234e445a8e4479a4d7f61da2" is_pre_election: true
I20260806 22:45:23.134804 3924 raft_consensus.cc:2454] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41dc974f6da145f3a63842c794d027e0 in term 0.
I20260806 22:45:23.135035 3711 leader_election.cc:302] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41dc974f6da145f3a63842c794d027e0, af444ef9234e445a8e4479a4d7f61da2; no voters:
I20260806 22:45:23.135191 3835 raft_consensus.cc:2781] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:23.135258 3835 raft_consensus.cc:482] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:23.135284 3835 raft_consensus.cc:3037] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:23.135397 4066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2bd57b5d0a84ae9851252bb24848c26" candidate_uuid: "41dc974f6da145f3a63842c794d027e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0bfc169191452b94a504aec98cf82c" is_pre_election: true
I20260806 22:45:23.135547 4066 raft_consensus.cc:2454] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41dc974f6da145f3a63842c794d027e0 in term 0.
I20260806 22:45:23.136082 3835 raft_consensus.cc:504] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:23.136250 3835 leader_election.cc:288] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 1 election: Requested vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
I20260806 22:45:23.136394 3924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2bd57b5d0a84ae9851252bb24848c26" candidate_uuid: "41dc974f6da145f3a63842c794d027e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af444ef9234e445a8e4479a4d7f61da2"
I20260806 22:45:23.136420 4066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2bd57b5d0a84ae9851252bb24848c26" candidate_uuid: "41dc974f6da145f3a63842c794d027e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0bfc169191452b94a504aec98cf82c"
I20260806 22:45:23.136484 4066 raft_consensus.cc:3037] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:23.136489 3924 raft_consensus.cc:3037] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:23.137158 4066 raft_consensus.cc:2454] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41dc974f6da145f3a63842c794d027e0 in term 1.
I20260806 22:45:23.137295 3924 raft_consensus.cc:2454] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41dc974f6da145f3a63842c794d027e0 in term 1.
I20260806 22:45:23.137347 3711 leader_election.cc:302] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41dc974f6da145f3a63842c794d027e0, bf0bfc169191452b94a504aec98cf82c; no voters:
I20260806 22:45:23.137441 3835 raft_consensus.cc:2781] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:23.137585 3835 raft_consensus.cc:686] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 LEADER]: Becoming Leader. State: Replica: 41dc974f6da145f3a63842c794d027e0, State: Running, Role: LEADER
I20260806 22:45:23.137671 3835 consensus_queue.cc:237] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:23.138477 3634 catalog_manager.cc:5495] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41dc974f6da145f3a63842c794d027e0 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "41dc974f6da145f3a63842c794d027e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } health_report { overall_health: HEALTHY } } }
I20260806 22:45:23.170084 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.168132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.19.74.193:45911)
0806 22:45:23.168188 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:23.168190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:23.168198 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:45:23.168254 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:23.168255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:23.168273 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:23.168339 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:23.168441 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.168910 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.169525 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.169695 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.169741 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.169860 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:23.169908 (+ 48us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 22:45:23.170009 (+ 101us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58770: token expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 22:45:23.170166 3837 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58770: token expired
I20260806 22:45:23.175977 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.170478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.19.74.254:38903)
0806 22:45:23.170517 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:23.170519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:23.170526 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:45:23.170551 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:23.170551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:23.170558 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:23.170600 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:23.171086 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.171486 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.172125 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.172312 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.172361 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.173609 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:23.173610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:23.173610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:23.173615 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:23.175029 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:23.175320 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:23.175320 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:23.175321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:23.175369 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:23.175584 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:23.175585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:23.175585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:23.175697 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:23.175767 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:23.175858 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:45:23.175919 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:45:23.179136 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.176962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58784 (local address 127.19.74.193:45911)
0806 22:45:23.177008 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:23.177010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:23.177016 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:45:23.177085 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:23.177085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:23.177093 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:23.177138 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:23.177300 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.177763 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.178511 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.178722 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.178773 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.178883 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:23.178956 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:23.178989 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:23.179027 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:45:23.179052 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:45:23.180625 4066 raft_consensus.cc:1261] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 1 FOLLOWER]: Refusing update from remote peer 41dc974f6da145f3a63842c794d027e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:23.180626 3924 raft_consensus.cc:1261] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 1 FOLLOWER]: Refusing update from remote peer 41dc974f6da145f3a63842c794d027e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:23.180975 3835 consensus_queue.cc:1035] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:23.181134 4132 consensus_queue.cc:1035] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:23.184242 4137 mvcc.cc:205] Tried to move back new op lower bound from 7315686699745460224 to 7315686699572346880. Current Snapshot: MvccSnapshot[applied={T|T < 7315686699745460224 or (T in {7315686699745460224})}]
I20260806 22:45:23.184754 4136 mvcc.cc:205] Tried to move back new op lower bound from 7315686699745460224 to 7315686699572346880. Current Snapshot: MvccSnapshot[applied={T|T < 7315686699745460224}]
I20260806 22:45:23.185657 3634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:23.185745 3634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:23.186481 3634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Sending DeleteTablet for 3 replicas of tablet f2bd57b5d0a84ae9851252bb24848c26
I20260806 22:45:23.186802 3760 tablet_service.cc:1514] Processing DeleteTablet for tablet f2bd57b5d0a84ae9851252bb24848c26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:23 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:58754
I20260806 22:45:23.186856 3904 tablet_service.cc:1514] Processing DeleteTablet for tablet f2bd57b5d0a84ae9851252bb24848c26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:23 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:49016
I20260806 22:45:23.186908 3838 tablet_replica.cc:330] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: stopping tablet replica
I20260806 22:45:23.186852 4045 tablet_service.cc:1514] Processing DeleteTablet for tablet f2bd57b5d0a84ae9851252bb24848c26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:23 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:41160
I20260806 22:45:23.186959 3838 raft_consensus.cc:2229] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:23.187047 3838 raft_consensus.cc:2258] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:23.187199 4145 tablet_replica.cc:330] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: stopping tablet replica
I20260806 22:45:23.187264 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 3674
I20260806 22:45:23.187299 4145 raft_consensus.cc:2229] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:23.187337 3838 ts_tablet_manager.cc:1905] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:23.188570 4146 tablet_replica.cc:330] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: stopping tablet replica
I20260806 22:45:23.188668 4146 raft_consensus.cc:2229] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:23.188755 4146 raft_consensus.cc:2258] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:23.189216 4146 ts_tablet_manager.cc:1905] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:23.191080 4145 raft_consensus.cc:2258] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:23.191249 4146 ts_tablet_manager.cc:1918] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:23.191320 4146 log.cc:1196] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/f2bd57b5d0a84ae9851252bb24848c26
I20260806 22:45:23.191524 4145 ts_tablet_manager.cc:1905] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:23.191617 4146 ts_tablet_manager.cc:1939] T f2bd57b5d0a84ae9851252bb24848c26 P bf0bfc169191452b94a504aec98cf82c: Deleting consensus metadata
I20260806 22:45:23.192044 3617 catalog_manager.cc:4841] TS bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083): tablet f2bd57b5d0a84ae9851252bb24848c26 (table test-table [id=0e3d8368223849cc97bc1bd025f0f581]) successfully deleted
I20260806 22:45:23.193132 4145 ts_tablet_manager.cc:1918] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:23.193220 4145 log.cc:1196] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/f2bd57b5d0a84ae9851252bb24848c26
I20260806 22:45:23.193475 4145 ts_tablet_manager.cc:1939] T f2bd57b5d0a84ae9851252bb24848c26 P af444ef9234e445a8e4479a4d7f61da2: Deleting consensus metadata
I20260806 22:45:23.193919 3617 catalog_manager.cc:4841] TS af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563): tablet f2bd57b5d0a84ae9851252bb24848c26 (table test-table [id=0e3d8368223849cc97bc1bd025f0f581]) successfully deleted
W20260806 22:45:23.195299 3619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:45911 (error 108)
I20260806 22:45:23.195629 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 3834
I20260806 22:45:23.195818 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.195682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:45911 (local address 127.0.0.1:58786)
0806 22:45:23.195738 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:23.195783 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:45911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 22:45:23.196306 3619 catalog_manager.cc:4584] TS 41dc974f6da145f3a63842c794d027e0 (127.19.74.193:45911): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2bd57b5d0a84ae9851252bb24848c26: Network error: Client connection negotiation failed: client connection to 127.19.74.193:45911: connect: Connection refused (error 111)
I20260806 22:45:23.201920 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 3978
I20260806 22:45:23.207072 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 3599
I20260806 22:45:43.213696 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38903
--webserver_interface=127.19.74.254
--webserver_port=36077
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:43.299266 4148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:43.299468 4148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:43.299490 4148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:43.300621 4148 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:45:43.300660 4148 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:45:43.300673 4148 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:45:43.300714 4148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:43.300732 4148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:43.300756 4148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:43.300776 4148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:43.300804 4148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:43.302270 4148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=36077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:43.302520 4148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:43.302826 4148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:43.308069 4156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:43.308072 4154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:43.308230 4148 server_base.cc:1034] running on GCE node
W20260806 22:45:43.308148 4153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:43.308512 4148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:43.308799 4148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:43.309937 4148 hybrid_clock.cc:648] HybridClock initialized: now 1786056343309916 us; error 41 us; skew 500 ppm
Aug 06 22:45:43 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:43.318575 4148 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:45:43.319126 4148 webserver.cc:460] Webserver started at http://127.19.74.254:36077/ using document root <none> and password file <none>
I20260806 22:45:43.319381 4148 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:43.319427 4148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:43.320770 4148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:43.321468 4163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.321640 4148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:45:43.321753 4148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "325b9f4c914d4c24af9a021d3bef614d"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "b136bf4b406649d02cc4ba91c438532a"
server_key_iv: "38961aef9755ca7d5055520274bb47e6"
server_key_version: "encryptionkey@0"
I20260806 22:45:43.322142 4148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:43.354614 4148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:43.356504 4148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:43.356676 4148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:43.361055 4148 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:38903
I20260806 22:45:43.361109 4217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:38903 every 8 connection(s)
I20260806 22:45:43.361557 4148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:43.363121 4218 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:45:43.363776 4218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Bootstrap starting.
I20260806 22:45:43.370566 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 4148
I20260806 22:45:43.370925 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:45911
--local_ip_for_outbound_sockets=127.19.74.193
--tserver_master_addrs=127.19.74.254:38903
--webserver_port=38925
--webserver_interface=127.19.74.193
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:45:43.373021 4218 log.cc:824] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:43.375461 4218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:45:43.375746 4218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Bootstrap complete.
I20260806 22:45:43.377952 4218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:43.378168 4218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325b9f4c914d4c24af9a021d3bef614d, State: Initialized, Role: FOLLOWER
I20260806 22:45:43.378356 4218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:43.378437 4218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:43.378490 4218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:43.378563 4218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:45:43.379554 4218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:43.379726 4218 leader_election.cc:302] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 325b9f4c914d4c24af9a021d3bef614d; no voters:
I20260806 22:45:43.379940 4218 leader_election.cc:288] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:45:43.379974 4222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:45:43.380118 4222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [term 3 LEADER]: Becoming Leader. State: Replica: 325b9f4c914d4c24af9a021d3bef614d, State: Running, Role: LEADER
I20260806 22:45:43.380206 4222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } }
I20260806 22:45:43.380242 4218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:43.380496 4223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "325b9f4c914d4c24af9a021d3bef614d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } } }
I20260806 22:45:43.380628 4223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:43.380493 4222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 325b9f4c914d4c24af9a021d3bef614d. Latest consensus state: current_term: 3 leader_uuid: "325b9f4c914d4c24af9a021d3bef614d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b9f4c914d4c24af9a021d3bef614d" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38903 } } }
I20260806 22:45:43.380836 4222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:43.381166 4227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:43.381992 4227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:43.382093 4227 catalog_manager.cc:1259] Loaded cluster ID: e582665784b64360b1f322694323a5a2
I20260806 22:45:43.382119 4227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:43.383487 4227 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:43.383694 4227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Loaded TSK: 0
I20260806 22:45:43.383747 4227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 325b9f4c914d4c24af9a021d3bef614d: Loaded TSK: 1
F20260806 22:45:43.383971 4227 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7ea6d119ad google::LogMessage::Fail() at ??:0
@ 0x7f7ea6d1488a google::LogMessage::SendToLog() at ??:0
@ 0x7f7ea6d114de google::LogMessage::Flush() at ??:0
@ 0x7f7ea6d12a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7ea947e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f7ea947bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f7eaec6f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f7eaec6f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f7eaec6f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f7eaec35c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f7eaec3fad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7ea8bdf0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7ea8bd4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7ea9a386db start_thread at ??:0
@ 0x7f7ea580f61f clone at ??:0
W20260806 22:45:43.476655 4220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:43.476840 4220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:43.476862 4220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:43.477995 4220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:43.478036 4220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:43.478062 4220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:45:43.479562 4220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:45911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=38925
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:43.479808 4220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:43.480058 4220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:43.486068 4261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:43.486121 4260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:43.486068 4263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:43.486367 4220 server_base.cc:1034] running on GCE node
I20260806 22:45:43.486687 4220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:43.486945 4220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:43.488114 4220 hybrid_clock.cc:648] HybridClock initialized: now 1786056343488098 us; error 34 us; skew 500 ppm
Aug 06 22:45:43 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:43.490886 4220 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:45:43.491349 4220 webserver.cc:460] Webserver started at http://127.19.74.193:38925/ using document root <none> and password file <none>
I20260806 22:45:43.491598 4220 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:43.491668 4220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:43.493049 4220 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:43.493889 4270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.494134 4220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:43.494269 4220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "41dc974f6da145f3a63842c794d027e0"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "aa7647354dafd39067c2771240e0e4e2"
server_key_iv: "b05d34fe6f97d6cb84191bc804997b59"
server_key_version: "encryptionkey@0"
I20260806 22:45:43.494711 4220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:43.506983 4220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:43.509265 4220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:43.509428 4220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:43.509661 4220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:43.510224 4278 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:45:43.512224 4220 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:45:43.512312 4220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:45:43.512372 4220 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:45:43.513033 4220 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:45:43.513093 4220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:43.513157 4278 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Bootstrap starting.
I20260806 22:45:43.515069 4278 log.cc:824] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:43.518602 4278 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:45:43.518996 4278 tablet_bootstrap.cc:492] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Bootstrap complete.
I20260806 22:45:43.519210 4278 ts_tablet_manager.cc:1397] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 22:45:43.520510 4220 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:45911
I20260806 22:45:43.520902 4220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:43.521232 4389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:45911 every 8 connection(s)
I20260806 22:45:43.521397 4278 raft_consensus.cc:348] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:43.522250 4278 raft_consensus.cc:729] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41dc974f6da145f3a63842c794d027e0, State: Initialized, Role: FOLLOWER
I20260806 22:45:43.522399 4278 consensus_queue.cc:260] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:43.522699 4278 ts_tablet_manager.cc:1428] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:45:43.528213 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 4220
I20260806 22:45:43.528518 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:39563
--local_ip_for_outbound_sockets=127.19.74.194
--tserver_master_addrs=127.19.74.254:38903
--webserver_port=40823
--webserver_interface=127.19.74.194
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:43.584964 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:43.521842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:60843)
0806 22:45:43.522033 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.522078 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:43.522222 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:43.584749 (+ 62527us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":114,"threads_started":1}
W20260806 22:45:43.585315 4390 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:38903 (0 consecutive failures): Network error: Failed to ping master at 127.19.74.254:38903: Client connection negotiation failed: client connection to 127.19.74.254:38903: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:45:43.585706 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:43.585565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:37167)
0806 22:45:43.585635 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.585676 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:45:43.586079 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:43.585989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:43705)
0806 22:45:43.586040 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.586063 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 22:45:43.586215 4390 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:45:43.615978 4394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:43.616200 4394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:43.616238 4394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:43.617312 4394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:43.617372 4394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:43.617413 4394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:45:43.618860 4394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:39563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=40823
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:43.619110 4394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:43.619325 4394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:43.624305 4402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:43.624289 4400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:43.624306 4399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:43.624455 4394 server_base.cc:1034] running on GCE node
I20260806 22:45:43.624666 4394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:43.624859 4394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:43.625991 4394 hybrid_clock.cc:648] HybridClock initialized: now 1786056343625980 us; error 27 us; skew 500 ppm
Aug 06 22:45:43 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:43.628379 4394 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:45:43.628798 4394 webserver.cc:460] Webserver started at http://127.19.74.194:40823/ using document root <none> and password file <none>
I20260806 22:45:43.629030 4394 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:43.629076 4394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:43.630411 4394 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:43.631036 4409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.631207 4394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.631320 4394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "af444ef9234e445a8e4479a4d7f61da2"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "1ab88d4df3db650a2c848c9d9c7b65cb"
server_key_iv: "7906755dce9569eaedf3805a55ea907d"
server_key_version: "encryptionkey@0"
I20260806 22:45:43.631733 4394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:43.650677 4394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:43.652415 4394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:43.652571 4394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:43.652794 4394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:43.653090 4394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:43.653148 4394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.653205 4394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:43.653232 4394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.659296 4394 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:39563
I20260806 22:45:43.659350 4529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:39563 every 8 connection(s)
I20260806 22:45:43.659732 4394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:43.660759 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:43.660397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:54009)
0806 22:45:43.660551 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.660623 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260806 22:45:43.661181 4530 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:38903 (0 consecutive failures): Network error: Failed to ping master at 127.19.74.254:38903: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
I20260806 22:45:43.661442 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:43.661358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:51957)
0806 22:45:43.661399 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.661425 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:45:43.661700 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:43.661644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:34821)
0806 22:45:43.661668 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.661687 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260806 22:45:43.661813 4530 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:45:43.664618 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 4394
I20260806 22:45:43.664891 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:36083
--local_ip_for_outbound_sockets=127.19.74.195
--tserver_master_addrs=127.19.74.254:38903
--webserver_port=39539
--webserver_interface=127.19.74.195
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:43.760965 4533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:43.761140 4533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:43.761161 4533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:43.762135 4533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:43.762164 4533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:43.762185 4533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:45:43.763505 4533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:36083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=39539
--tserver_master_addrs=127.19.74.254:38903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:45:43.763700 4533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:43.763860 4533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:43.768858 4539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:43.768894 4533 server_base.cc:1034] running on GCE node
W20260806 22:45:43.768898 4538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:43.768855 4542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:43.769169 4533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:43.769354 4533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:43.770490 4533 hybrid_clock.cc:648] HybridClock initialized: now 1786056343770482 us; error 25 us; skew 500 ppm
Aug 06 22:45:43 dist-test-slave-pgkr krb5kdc[3033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:43.773141 4533 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:45:43.773556 4533 webserver.cc:460] Webserver started at http://127.19.74.195:39539/ using document root <none> and password file <none>
I20260806 22:45:43.773809 4533 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:43.773854 4533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:43.775230 4533 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:43.775816 4550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.776033 4533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.776152 4533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "bf0bfc169191452b94a504aec98cf82c"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-pgkr"
server_key: "a57195009afa283fcc523e6fffb48d0d"
server_key_iv: "4f31c2d5ca967c1a422b419f00d58b9d"
server_key_version: "encryptionkey@0"
I20260806 22:45:43.776508 4533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:43.787222 4533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:43.789436 4533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:43.789616 4533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:43.789836 4533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:43.790202 4533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:43.790251 4533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.790292 4533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:43.790323 4533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:43.796522 4533 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:36083
I20260806 22:45:43.796618 4668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:36083 every 8 connection(s)
I20260806 22:45:43.796934 4533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:43.797981 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:43.797573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:53015)
0806 22:45:43.797750 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.797823 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
W20260806 22:45:43.798352 4669 heartbeater.cc:644] Failed to heartbeat to 127.19.74.254:38903 (0 consecutive failures): Network error: Failed to ping master at 127.19.74.254:38903: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
I20260806 22:45:43.798666 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:43.798526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:55177)
0806 22:45:43.798574 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.798605 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:45:43.798933 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:43.798852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:35061)
0806 22:45:43.798894 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:43.798915 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:45:43.799036 4669 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:45:43.800609 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 4533
I20260806 22:45:44.586906 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:44.586683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:49601)
0806 22:45:44.586795 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:44.586824 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260806 22:45:44.662489 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:44.662215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:40307)
0806 22:45:44.662412 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:44.662445 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":59,"threads_started":1}
I20260806 22:45:44.799503 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:44.799294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:46913)
0806 22:45:44.799427 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:44.799456 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260806 22:45:44.815461 4676 raft_consensus.cc:482] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:44.815572 4676 raft_consensus.cc:504] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:44.815846 4676 leader_election.cc:288] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
Aug 06 22:45:44 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.195@KRBTEST.COM
Aug 06 22:45:44 dist-test-slave-pgkr krb5kdc[3033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.194@KRBTEST.COM
I20260806 22:45:44.827371 4678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:44.816201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:51471 (local address 127.19.74.194:39563)
0806 22:45:44.816386 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:44.816399 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:44.816437 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:45:44.817044 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:44.817046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:44.817099 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:44.817256 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:44.818060 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:44.819093 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:44.820170 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:44.820569 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:44.820690 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:44.823487 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:44.823490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:44.823490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:44.823499 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:44.825255 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:44.825808 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:44.825811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:44.825812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:44.825953 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:44.826305 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:44.826306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:44.826306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:44.826640 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:44.826726 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:44.827106 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 22:45:44.827245 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260806 22:45:44.827371 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:44.816298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:39625 (local address 127.19.74.195:36083)
0806 22:45:44.816445 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:44.816452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:44.816479 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:45:44.817033 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:44.817037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:44.817099 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:44.817256 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:44.818243 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:44.819210 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:44.820152 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:44.820572 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:44.820691 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:44.823086 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:44.823091 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:44.823092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:44.823104 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:44.825136 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:44.825808 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:44.825811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:44.825812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:44.825957 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:44.826305 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:44.826306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:44.826306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:44.826612 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:44.826726 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:44.827106 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 22:45:44.827245 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":49,"threads_started":1}
I20260806 22:45:44.827391 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:44.816177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:36083 (local address 127.19.74.193:39625)
0806 22:45:44.816346 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:44.816355 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:44.816402 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:44.817386 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:44.817387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:44.817669 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:44.818134 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:44.818137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:44.819306 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:44.819307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:44.820058 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:44.820061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:44.820666 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:44.820667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:44.820790 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:44.821446 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:44.821454 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:44.822953 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:44.825246 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:44.825254 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:44.825257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:44.825701 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:44.826048 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:44.826049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:44.826049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:44.826225 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:44.826711 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:44.826721 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:44.826885 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 22:45:44.827032 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 22:45:44.827235 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":3,"thread_start_us":70,"threads_started":1}
I20260806 22:45:44.827692 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:44.816064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:39563 (local address 127.19.74.193:51471)
0806 22:45:44.816148 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:44.816163 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:44.816241 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:44.817316 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:44.817319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:44.817682 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:44.817902 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:44.817908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:44.819191 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:44.819192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:44.820060 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:44.820062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:44.820664 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:44.820665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:44.820789 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:44.821463 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:44.821467 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:44.823367 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:44.825317 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:44.825318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:44.825318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:44.825701 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:44.826048 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:44.826049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:44.826049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:44.826225 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:44.826711 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:44.826721 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:44.826885 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 22:45:44.827032 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 22:45:44.827293 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":191}
W20260806 22:45:44.828763 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
W20260806 22:45:44.828862 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
I20260806 22:45:44.828933 4271 leader_election.cc:302] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41dc974f6da145f3a63842c794d027e0; no voters: af444ef9234e445a8e4479a4d7f61da2, bf0bfc169191452b94a504aec98cf82c
I20260806 22:45:44.829056 4676 raft_consensus.cc:2726] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:45:45.587513 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:45.587313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:48615)
0806 22:45:45.587434 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:45.587466 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260806 22:45:45.663201 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:45.663020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:57039)
0806 22:45:45.663135 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:45.663169 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260806 22:45:45.800156 4682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:45.799937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:33791)
0806 22:45:45.800082 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:45.800118 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260806 22:45:46.588270 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:46.588038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:50503)
0806 22:45:46.588184 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:46.588223 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260806 22:45:46.663826 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:46.663635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:59353)
0806 22:45:46.663753 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:46.663788 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260806 22:45:46.785794 4687 raft_consensus.cc:482] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:46.785899 4687 raft_consensus.cc:504] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:46.786077 4687 leader_election.cc:288] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
W20260806 22:45:46.786679 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
W20260806 22:45:46.786768 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
I20260806 22:45:46.786787 4271 leader_election.cc:302] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41dc974f6da145f3a63842c794d027e0; no voters: af444ef9234e445a8e4479a4d7f61da2, bf0bfc169191452b94a504aec98cf82c
I20260806 22:45:46.786895 4687 raft_consensus.cc:2726] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:45:46.800789 4688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:46.800577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:37751)
0806 22:45:46.800719 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:46.800752 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260806 22:45:47.588955 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:47.588741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:36773)
0806 22:45:47.588871 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:47.588908 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260806 22:45:47.664570 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:47.664262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:39765)
0806 22:45:47.664487 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:47.664525 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":72,"threads_started":1}
I20260806 22:45:47.801386 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:47.801174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:60421)
0806 22:45:47.801315 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:47.801348 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260806 22:45:48.589675 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:48.589441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:38253)
0806 22:45:48.589578 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:48.589629 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260806 22:45:48.665278 4693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:48.665078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:52965)
0806 22:45:48.665207 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:48.665240 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260806 22:45:48.801926 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:48.801745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:39643)
0806 22:45:48.801857 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:48.801891 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260806 22:45:49.590322 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:49.590102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:33223)
0806 22:45:49.590240 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:49.590273 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260806 22:45:49.665910 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:49.665690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:34573)
0806 22:45:49.665826 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:49.665859 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260806 22:45:49.802659 4697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:49.802395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:50337)
0806 22:45:49.802526 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:49.802560 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260806 22:45:49.954664 4698 raft_consensus.cc:482] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:49.954766 4698 raft_consensus.cc:504] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:49.954968 4698 leader_election.cc:288] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
W20260806 22:45:49.955402 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
W20260806 22:45:49.955484 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
I20260806 22:45:49.955521 4271 leader_election.cc:302] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41dc974f6da145f3a63842c794d027e0; no voters: af444ef9234e445a8e4479a4d7f61da2, bf0bfc169191452b94a504aec98cf82c
I20260806 22:45:49.955616 4698 raft_consensus.cc:2726] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:45:50.590962 4699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:50.590763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:52915)
0806 22:45:50.590879 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:50.590918 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260806 22:45:50.666690 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:50.666345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:50429)
0806 22:45:50.666550 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:50.666587 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":70,"threads_started":1}
I20260806 22:45:50.803253 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:50.803067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:44461)
0806 22:45:50.803184 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:50.803215 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260806 22:45:51.591662 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:51.591428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:55063)
0806 22:45:51.591570 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:51.591617 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260806 22:45:51.667325 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:51.667115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:57541)
0806 22:45:51.667250 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:51.667285 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260806 22:45:51.803864 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:51.803651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:35933)
0806 22:45:51.803782 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:51.803825 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260806 22:45:52.592314 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:52.592072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:55397)
0806 22:45:52.592203 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:52.592249 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260806 22:45:52.667959 4706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:52.667742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:58847)
0806 22:45:52.667881 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:52.667915 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260806 22:45:52.804486 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:52.804272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:41133)
0806 22:45:52.804415 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:52.804446 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260806 22:45:53.593138 4708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:53.592885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:38023)
0806 22:45:53.593034 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:53.593079 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260806 22:45:53.668696 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:53.668455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:48663)
0806 22:45:53.668588 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:53.668626 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260806 22:45:53.805105 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:53.804913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:54367)
0806 22:45:53.805037 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:53.805066 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260806 22:45:54.111537 4711 raft_consensus.cc:482] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:54.111649 4711 raft_consensus.cc:504] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:54.111831 4711 leader_election.cc:288] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
W20260806 22:45:54.112334 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
W20260806 22:45:54.112409 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
I20260806 22:45:54.112426 4271 leader_election.cc:302] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41dc974f6da145f3a63842c794d027e0; no voters: af444ef9234e445a8e4479a4d7f61da2, bf0bfc169191452b94a504aec98cf82c
I20260806 22:45:54.112498 4711 raft_consensus.cc:2726] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:45:54.593775 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:54.593553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:45375)
0806 22:45:54.593690 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:54.593726 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260806 22:45:54.669405 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:54.669179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:51751)
0806 22:45:54.669319 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:54.669364 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260806 22:45:54.805699 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:54.805511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:48367)
0806 22:45:54.805628 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:54.805658 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260806 22:45:55.594456 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:55.594253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:39033)
0806 22:45:55.594370 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:55.594410 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260806 22:45:55.669992 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:55.669796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:45241)
0806 22:45:55.669912 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:55.669948 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260806 22:45:55.806281 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:55.806080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:60607)
0806 22:45:55.806207 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:55.806237 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260806 22:45:56.233275 4718 raft_consensus.cc:482] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:56.233372 4718 raft_consensus.cc:504] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af444ef9234e445a8e4479a4d7f61da2" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 39563 } } peers { permanent_uuid: "bf0bfc169191452b94a504aec98cf82c" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36083 } } peers { permanent_uuid: "41dc974f6da145f3a63842c794d027e0" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 45911 } }
I20260806 22:45:56.233594 4718 leader_election.cc:288] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563), bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083)
W20260806 22:45:56.234045 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf0bfc169191452b94a504aec98cf82c (127.19.74.195:36083): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
W20260806 22:45:56.234166 4271 leader_election.cc:341] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af444ef9234e445a8e4479a4d7f61da2 (127.19.74.194:39563): Not found: Tablet not found: f2bd57b5d0a84ae9851252bb24848c26
I20260806 22:45:56.234194 4271 leader_election.cc:302] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41dc974f6da145f3a63842c794d027e0; no voters: af444ef9234e445a8e4479a4d7f61da2, bf0bfc169191452b94a504aec98cf82c
I20260806 22:45:56.234279 4718 raft_consensus.cc:2726] T f2bd57b5d0a84ae9851252bb24848c26 P 41dc974f6da145f3a63842c794d027e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:45:56.595178 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:56.594945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:48815)
0806 22:45:56.595084 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:56.595132 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260806 22:45:56.670729 4720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:56.670455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:44991)
0806 22:45:56.670597 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:56.670685 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260806 22:45:56.807052 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:56.806822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:59201)
0806 22:45:56.806972 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:56.807008 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260806 22:45:57.595968 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:57.595698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:39569)
0806 22:45:57.595882 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:57.595922 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":125,"threads_started":1}
I20260806 22:45:57.671327 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:57.671120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:42257)
0806 22:45:57.671251 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:57.671282 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260806 22:45:57.807663 4724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:57.807460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:49977)
0806 22:45:57.807586 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:57.807621 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260806 22:45:58.596832 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:58.596620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.193:34073)
0806 22:45:58.596748 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:58.596787 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260806 22:45:58.671940 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:58.671739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.194:42905)
0806 22:45:58.671871 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:58.671899 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260806 22:45:58.805649 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 4220
I20260806 22:45:58.808405 4727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:58.808197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38903 (local address 127.19.74.195:33925)
0806 22:45:58.808327 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:58.808366 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.254:38903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260806 22:45:58.813845 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 4394
I20260806 22:45:58.819694 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 4533
2026-08-06T22:45:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58630 ms)
[----------] 30 tests from SecurityITest (231170 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:58 dist-test-slave-pgkr krb5kdc[4732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:58 dist-test-slave-pgkr krb5kdc[4732](info): set up 2 sockets
Aug 06 22:45:58 dist-test-slave-pgkr krb5kdc[4732](info): commencing operation
krb5kdc: starting...
W20260806 22:45:59.887141 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:45:59 dist-test-slave-pgkr krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:00.009135 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:43585
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:43585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:00.092823 4747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:00.093026 4747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:00.093050 4747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:00.094273 4747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:00.094316 4747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:00.094329 4747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:00.094341 4747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:00.094353 4747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:00.095826 4747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:43585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:43585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:00.096084 4747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:00.096328 4747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:00.101257 4754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:00.101265 4756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:00.101279 4753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:00.101651 4747 server_base.cc:1034] running on GCE node
I20260806 22:46:00.101868 4747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:00.102141 4747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:00.103283 4747 hybrid_clock.cc:648] HybridClock initialized: now 1786056360103270 us; error 28 us; skew 500 ppm
Aug 06 22:46:00 dist-test-slave-pgkr krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056360, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:00.105840 4747 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:00.106324 4747 webserver.cc:460] Webserver started at http://127.19.74.254:39475/ using document root <none> and password file <none>
I20260806 22:46:00.106550 4747 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:00.106598 4747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:00.106860 4747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:00.107931 4747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "de1952a3cf3246b5934ccbb81ecf9b13"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "82986d2b7dbc6fce765b904a20575b52"
server_key_iv: "a04d4bf6c6994c0fd065cc2d5b10224e"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.108304 4747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "de1952a3cf3246b5934ccbb81ecf9b13"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "82986d2b7dbc6fce765b904a20575b52"
server_key_iv: "a04d4bf6c6994c0fd065cc2d5b10224e"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.109619 4747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:00.110375 4763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.110563 4747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:00.110693 4747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "de1952a3cf3246b5934ccbb81ecf9b13"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "82986d2b7dbc6fce765b904a20575b52"
server_key_iv: "a04d4bf6c6994c0fd065cc2d5b10224e"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.110766 4747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:00.124281 4747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:00.126269 4747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:00.126416 4747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:00.130568 4747 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:43585
I20260806 22:46:00.130599 4816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:43585 every 8 connection(s)
I20260806 22:46:00.131091 4747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:00.132002 4818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:00.134423 4818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13: Bootstrap starting.
I20260806 22:46:00.134443 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 4747
I20260806 22:46:00.134562 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:00.134817 19755 external_mini_cluster.cc:1428] Setting key a8b24701579645e45c71ba600a7d7178
I20260806 22:46:00.135177 4818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:00.135564 4818 log.cc:824] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:00.136358 4818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13: No bootstrap required, opened a new log
I20260806 22:46:00.138707 4818 raft_consensus.cc:348] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1952a3cf3246b5934ccbb81ecf9b13" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43585 } }
I20260806 22:46:00.138881 4818 raft_consensus.cc:374] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:00.138908 4818 raft_consensus.cc:729] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1952a3cf3246b5934ccbb81ecf9b13, State: Initialized, Role: FOLLOWER
I20260806 22:46:00.139061 4818 consensus_queue.cc:260] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1952a3cf3246b5934ccbb81ecf9b13" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43585 } }
I20260806 22:46:00.139127 4818 raft_consensus.cc:388] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:00.139147 4818 raft_consensus.cc:482] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:00.139211 4818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:00.139820 4818 raft_consensus.cc:504] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1952a3cf3246b5934ccbb81ecf9b13" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43585 } }
I20260806 22:46:00.139927 4818 leader_election.cc:302] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de1952a3cf3246b5934ccbb81ecf9b13; no voters:
I20260806 22:46:00.140077 4818 leader_election.cc:288] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:00.140139 4824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:00.140311 4824 raft_consensus.cc:686] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [term 1 LEADER]: Becoming Leader. State: Replica: de1952a3cf3246b5934ccbb81ecf9b13, State: Running, Role: LEADER
I20260806 22:46:00.140347 4818 sys_catalog.cc:564] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:00.140414 4824 consensus_queue.cc:237] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1952a3cf3246b5934ccbb81ecf9b13" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43585 } }
I20260806 22:46:00.140762 4824 sys_catalog.cc:455] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de1952a3cf3246b5934ccbb81ecf9b13. Latest consensus state: current_term: 1 leader_uuid: "de1952a3cf3246b5934ccbb81ecf9b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1952a3cf3246b5934ccbb81ecf9b13" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43585 } } }
I20260806 22:46:00.140910 4824 sys_catalog.cc:458] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:00.140977 4825 sys_catalog.cc:455] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de1952a3cf3246b5934ccbb81ecf9b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1952a3cf3246b5934ccbb81ecf9b13" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 43585 } } }
I20260806 22:46:00.141091 4825 sys_catalog.cc:458] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:00.141263 4833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:00.142050 4833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:46:00 dist-test-slave-pgkr krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:00.146718 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.135407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.19.74.254:43585)
0806 22:46:00.135618 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.135629 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.135664 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:46:00.136285 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.136295 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.136357 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.136514 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:00.137166 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.138678 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.139593 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.140228 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.140694 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.143513 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:00.143517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:00.143517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:00.143529 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:00.145156 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.145523 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.145526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.145527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.145622 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.145859 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.145860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.145861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.146091 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:00.146270 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.146352 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.146490 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
I20260806 22:46:00.147267 4833 catalog_manager.cc:1347] Generated new cluster ID: 72815ffc9cdb4da883be798473715a95
I20260806 22:46:00.147322 4833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:00.165518 4833 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:00.166106 4833 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:00.170156 4833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13: Generated new TSK 0
I20260806 22:46:00.170364 4833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:46:00.205003 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43585
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:00.292493 4846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:00.292701 4846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:00.292722 4846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:00.293747 4846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:00.293777 4846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:00.293800 4846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:46:00.295086 4846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:00.295257 4846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:00.295444 4846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:00.300376 4854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:00.300374 4852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:00.300405 4846 server_base.cc:1034] running on GCE node
W20260806 22:46:00.300376 4851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:00.300845 4846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:00.301057 4846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:00.302192 4846 hybrid_clock.cc:648] HybridClock initialized: now 1786056360302178 us; error 28 us; skew 500 ppm
Aug 06 22:46:00 dist-test-slave-pgkr krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056360, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:00.304606 4846 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:46:00.305013 4846 webserver.cc:460] Webserver started at http://127.19.74.193:35051/ using document root <none> and password file <none>
I20260806 22:46:00.305254 4846 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:00.305322 4846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:00.305465 4846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:00.306463 4846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "7d510068c7ef456d9694ddb21522cd9e"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "3a5f0872dbff72422abaf8cf56da17f9"
server_key_iv: "fc2a8ee56389339cfdca745e2e9991f2"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.306880 4846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "7d510068c7ef456d9694ddb21522cd9e"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "3a5f0872dbff72422abaf8cf56da17f9"
server_key_iv: "fc2a8ee56389339cfdca745e2e9991f2"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.308171 4846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:00.308868 4861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.309032 4846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:00.309135 4846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "7d510068c7ef456d9694ddb21522cd9e"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "3a5f0872dbff72422abaf8cf56da17f9"
server_key_iv: "fc2a8ee56389339cfdca745e2e9991f2"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.309216 4846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:00.326503 4846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:00.328248 4846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:00.328410 4846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:00.328644 4846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:00.328945 4846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:00.328997 4846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.329030 4846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:00.329056 4846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.334924 4846 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:40753
I20260806 22:46:00.335033 4978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:40753 every 8 connection(s)
I20260806 22:46:00.335302 4846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:00.340752 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 4846
I20260806 22:46:00.340862 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
Aug 06 22:46:00 dist-test-slave-pgkr krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056360, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:00.341073 19755 external_mini_cluster.cc:1428] Setting key 10752258f1d558680090d2e57cf03dd3
I20260806 22:46:00.344419 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.336013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:43585 (local address 127.19.74.193:60563)
0806 22:46:00.336188 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.336227 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:00.336688 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:00.336891 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:00.336894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:00.337438 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:00.337639 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.337644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.338270 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.338271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.339109 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.339112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.339462 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.339463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.339556 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.340016 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:00.340024 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:00.341450 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:00.343008 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:00.343016 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:00.343018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:00.343334 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:00.343522 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:00.343523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:00.343523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:00.343670 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:00.343960 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:00.343965 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:00.344070 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 22:46:00.344163 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:46:00.344274 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260806 22:46:00.344419 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.336068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:60563 (local address 127.19.74.254:43585)
0806 22:46:00.336139 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.336141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.336311 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:46:00.336772 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.336773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.336782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.336834 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:00.337713 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.338188 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.339184 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.339390 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.339441 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.341526 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:00.341527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:00.341527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:00.341534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:00.342934 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.343408 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.343409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.343409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.343458 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.343746 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.343747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.343747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.343888 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:00.343960 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.344206 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.344297 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:46:00.345170 4979 heartbeater.cc:347] Connected to a master server at 127.19.74.254:43585
I20260806 22:46:00.345271 4979 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:00.345449 4979 heartbeater.cc:511] Master 127.19.74.254:43585 requested a full tablet report, sending...
I20260806 22:46:00.345963 4780 ts_manager.cc:194] Registered new tserver with Master: 7d510068c7ef456d9694ddb21522cd9e (127.19.74.193:40753)
I20260806 22:46:00.346570 4780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:60563
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:46:00.368860 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43585
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:00.451752 4984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:00.451948 4984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:00.451972 4984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:00.453050 4984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:00.453087 4984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:00.453119 4984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:46:00.454559 4984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:00.454768 4984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:00.454952 4984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:00.459633 4990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:00.459697 4984 server_base.cc:1034] running on GCE node
W20260806 22:46:00.459674 4993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:00.459651 4989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:00.460012 4984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:00.460263 4984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:00.461421 4984 hybrid_clock.cc:648] HybridClock initialized: now 1786056360461406 us; error 26 us; skew 500 ppm
Aug 06 22:46:00 dist-test-slave-pgkr krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056360, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:00.463965 4984 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:46:00.464435 4984 webserver.cc:460] Webserver started at http://127.19.74.194:38787/ using document root <none> and password file <none>
I20260806 22:46:00.464679 4984 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:00.464746 4984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:00.464895 4984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:00.465957 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "05b4e6d3855d41a09995fc3673950cd1"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "19d53c46b558c3a1c993dd5e5acbecd3"
server_key_iv: "352bc2bdfebc0d2063fe927f0bfc5315"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.466375 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "05b4e6d3855d41a09995fc3673950cd1"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "19d53c46b558c3a1c993dd5e5acbecd3"
server_key_iv: "352bc2bdfebc0d2063fe927f0bfc5315"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.467770 4984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:00.468464 5001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.468658 4984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:00.468765 4984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "05b4e6d3855d41a09995fc3673950cd1"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "19d53c46b558c3a1c993dd5e5acbecd3"
server_key_iv: "352bc2bdfebc0d2063fe927f0bfc5315"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.468837 4984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:00.479776 4984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:00.481513 4984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:00.481690 4984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:00.481911 4984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:00.482240 4984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:00.482295 4984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.482331 4984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:00.482359 4984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.488564 4984 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:35119
I20260806 22:46:00.488626 5119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:35119 every 8 connection(s)
I20260806 22:46:00.488930 4984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:00.494305 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 4984
I20260806 22:46:00.494405 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:00.494709 19755 external_mini_cluster.cc:1428] Setting key 33ff166c9f72e98be3b9f77470e1c6f9
Aug 06 22:46:00 dist-test-slave-pgkr krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056360, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:00.499164 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.489669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:41371 (local address 127.19.74.254:43585)
0806 22:46:00.489733 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.489735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.489918 (+ 183us) server_negotiation.cc:407] Connection header received
0806 22:46:00.490435 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.490437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.490454 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.490540 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:00.491568 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.492074 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.493161 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.493363 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.493450 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.495952 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:00.495954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:00.495954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:00.495963 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:00.497661 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.498164 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.498164 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.498165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.498214 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.498492 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.498492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.498492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.498592 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:00.498702 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.499008 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.499083 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:46:00.499470 5122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.489650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:43585 (local address 127.19.74.194:41371)
0806 22:46:00.489800 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.489835 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:00.490331 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:00.490647 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:00.490651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:00.491239 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:00.491429 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.491434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.492169 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.492170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.493051 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.493054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.493470 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.493471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.493598 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.494134 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:00.494143 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:00.495862 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:00.497742 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:00.497748 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:00.497751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:00.498094 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:00.498287 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:00.498287 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:00.498288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:00.498428 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:00.498694 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:00.498700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:00.498870 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:46:00.498986 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:46:00.499191 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260806 22:46:00.500085 5120 heartbeater.cc:347] Connected to a master server at 127.19.74.254:43585
I20260806 22:46:00.500211 5120 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:00.500427 5120 heartbeater.cc:511] Master 127.19.74.254:43585 requested a full tablet report, sending...
I20260806 22:46:00.500854 4780 ts_manager.cc:194] Registered new tserver with Master: 05b4e6d3855d41a09995fc3673950cd1 (127.19.74.194:35119)
I20260806 22:46:00.501242 4780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:41371
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:46:00.521870 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43585
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:00.602988 5127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:00.603173 5127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:00.603194 5127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:00.604161 5127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:00.604192 5127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:00.604214 5127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:46:00.605535 5127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:43585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:00.605716 5127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:00.605916 5127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:00.610693 5132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:00.610692 5135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:00.610692 5133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:00.610986 5127 server_base.cc:1034] running on GCE node
I20260806 22:46:00.611167 5127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:00.611392 5127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:00.612556 5127 hybrid_clock.cc:648] HybridClock initialized: now 1786056360612547 us; error 26 us; skew 500 ppm
Aug 06 22:46:00 dist-test-slave-pgkr krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056360, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:00.615151 5127 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:46:00.615571 5127 webserver.cc:460] Webserver started at http://127.19.74.195:34309/ using document root <none> and password file <none>
I20260806 22:46:00.615751 5127 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:00.615793 5127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:00.615898 5127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:00.616822 5127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "fb428ca3f1fa40b29fb483aeecdce8ff"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "a3735df9b6963cb1f7c35998b206ea31"
server_key_iv: "13da94246b6bbf5e18fe950d7cb54557"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.617122 5127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "fb428ca3f1fa40b29fb483aeecdce8ff"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "a3735df9b6963cb1f7c35998b206ea31"
server_key_iv: "13da94246b6bbf5e18fe950d7cb54557"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.618343 5127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:00.619060 5142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.619236 5127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.619330 5127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "fb428ca3f1fa40b29fb483aeecdce8ff"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "a3735df9b6963cb1f7c35998b206ea31"
server_key_iv: "13da94246b6bbf5e18fe950d7cb54557"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.619403 5127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:00.635095 5127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:00.637501 5127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:00.637665 5127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:00.637898 5127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:00.638204 5127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:00.638237 5127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.638278 5127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:00.638305 5127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.644604 5127 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:34487
I20260806 22:46:00.644644 5258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:34487 every 8 connection(s)
I20260806 22:46:00.644956 5127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:00.646806 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 5127
I20260806 22:46:00.646927 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:00.647148 19755 external_mini_cluster.cc:1428] Setting key 895977d39cbc169bdde973b2982cc01b
Aug 06 22:46:00 dist-test-slave-pgkr krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056360, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:00.653988 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.645657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:55757 (local address 127.19.74.254:43585)
0806 22:46:00.645698 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.645700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.645898 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:46:00.646342 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.646343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.646353 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.646413 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:00.647715 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.648413 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.649363 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.649547 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.649605 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.651683 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:00.651684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:00.651685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:00.651691 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:00.652888 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.653240 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.653241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.653241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.653291 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.653506 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.653506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.653507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.653601 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:00.653658 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.653869 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.653927 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:46:00.654138 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.645611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:43585 (local address 127.19.74.195:55757)
0806 22:46:00.645764 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.645810 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:00.646264 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:00.646464 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:00.646467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:00.647309 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:00.647610 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.647616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.648496 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.648497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.649270 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.649272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.649632 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.649633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.649728 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.650227 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:00.650235 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:00.651606 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:00.652960 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:00.652966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:00.652968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:00.653176 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:00.653348 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:00.653349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:00.653349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:00.653454 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:00.653662 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:00.653666 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:00.653767 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 22:46:00.653843 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:46:00.653951 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260806 22:46:00.654654 5259 heartbeater.cc:347] Connected to a master server at 127.19.74.254:43585
I20260806 22:46:00.654786 5259 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:00.654964 5259 heartbeater.cc:511] Master 127.19.74.254:43585 requested a full tablet report, sending...
I20260806 22:46:00.655369 4780 ts_manager.cc:194] Registered new tserver with Master: fb428ca3f1fa40b29fb483aeecdce8ff (127.19.74.195:34487)
I20260806 22:46:00.655736 4780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:55757
I20260806 22:46:00.658881 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:46:00.665434 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.660126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.19.74.254:43585)
0806 22:46:00.660185 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.660187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.660298 (+ 111us) server_negotiation.cc:407] Connection header received
0806 22:46:00.660332 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.660333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.660340 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.660385 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:00.660863 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.661287 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.661918 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.662101 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.662150 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.663206 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:00.663207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:00.663207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:00.663213 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:00.664558 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.664831 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.664832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.664832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.664880 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:00.665030 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:00.665030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:00.665031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:00.665173 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:00.665244 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.665324 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.665378 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:46:00.667208 4780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:00.668819 4780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:00.675323 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.671131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.19.74.195:34487)
0806 22:46:00.671309 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.671313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.671324 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:00.671358 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.671360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.671375 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.671460 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:00.671611 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.672768 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.674047 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.674824 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.674893 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.674948 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.675186 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.675229 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260806 22:46:00.675316 5268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.670986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:40753 (local address 127.0.0.1:51966)
0806 22:46:00.671275 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.671285 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:00.671336 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:00.671759 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:00.671761 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:00.671763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:00.671804 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.671805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.672780 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.672781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.674100 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.674103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.675056 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.675057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.675123 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.675127 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:00.675174 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:46:00.675215 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":42,"thread_start_us":56,"threads_started":1}
I20260806 22:46:00.675316 5267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.670976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:34487 (local address 127.0.0.1:51420)
0806 22:46:00.671071 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.671108 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:00.671172 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:00.671467 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:00.671469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:00.671471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:00.671536 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.671537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.672866 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.672866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.673952 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.673956 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.674987 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.674988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.675069 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.675082 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:46:00.675156 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:46:00.675215 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":345,"thread_start_us":50,"threads_started":1}
I20260806 22:46:00.675556 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.671175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:35119 (local address 127.0.0.1:48306)
0806 22:46:00.671346 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.671355 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:00.671393 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:00.671780 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:00.671781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:00.671782 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:00.671871 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.671873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.673141 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.673142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.674235 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.674237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.675162 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.675163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.675234 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.675239 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:00.675290 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:46:00.675332 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":32,"threads_started":1}
I20260806 22:46:00.675694 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.671096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51966 (local address 127.19.74.193:40753)
0806 22:46:00.671504 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.671509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.671519 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:00.671578 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.671581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.671599 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.671670 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:00.671884 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.672671 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.674190 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.674918 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.674977 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.675027 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.675518 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.675588 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":84,"threads_started":1}
I20260806 22:46:00.676141 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.671349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.19.74.194:35119)
0806 22:46:00.671623 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.671628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.671637 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:00.671677 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.671679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.671693 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.671776 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:00.671946 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.673053 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.674323 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.674998 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.675058 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.675112 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.676011 (+ 899us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.676062 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":92,"threads_started":1}
I20260806 22:46:00.676066 5190 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3bd35071b646baa3e3842f546af713 (DEFAULT_TABLE table=test-table [id=9a5ff3839e0f4664a9007efe65b7e049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:00.676332 4910 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3bd35071b646baa3e3842f546af713 (DEFAULT_TABLE table=test-table [id=9a5ff3839e0f4664a9007efe65b7e049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:00.676467 5190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3bd35071b646baa3e3842f546af713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:00.676689 4910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3bd35071b646baa3e3842f546af713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:00.676748 5052 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3bd35071b646baa3e3842f546af713 (DEFAULT_TABLE table=test-table [id=9a5ff3839e0f4664a9007efe65b7e049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:00.677039 5052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3bd35071b646baa3e3842f546af713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:00.679183 5275 tablet_bootstrap.cc:492] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: Bootstrap starting.
I20260806 22:46:00.679766 5275 tablet_bootstrap.cc:654] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:00.680223 5277 tablet_bootstrap.cc:492] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1: Bootstrap starting.
I20260806 22:46:00.680356 5276 tablet_bootstrap.cc:492] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e: Bootstrap starting.
I20260806 22:46:00.680680 5275 log.cc:824] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:00.680910 5277 tablet_bootstrap.cc:654] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:00.680974 5276 tablet_bootstrap.cc:654] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:00.681425 5275 tablet_bootstrap.cc:492] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: No bootstrap required, opened a new log
I20260806 22:46:00.681511 5275 ts_tablet_manager.cc:1397] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:00.681603 5276 log.cc:824] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:00.681593 5277 log.cc:824] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:00.682386 5276 tablet_bootstrap.cc:492] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e: No bootstrap required, opened a new log
I20260806 22:46:00.682458 5276 ts_tablet_manager.cc:1397] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:00.682646 5277 tablet_bootstrap.cc:492] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1: No bootstrap required, opened a new log
I20260806 22:46:00.682708 5277 ts_tablet_manager.cc:1397] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:00.683450 5275 raft_consensus.cc:348] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.683653 5275 raft_consensus.cc:374] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:00.683702 5275 raft_consensus.cc:729] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb428ca3f1fa40b29fb483aeecdce8ff, State: Initialized, Role: FOLLOWER
I20260806 22:46:00.683815 5275 consensus_queue.cc:260] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.684103 5259 heartbeater.cc:503] Master 127.19.74.254:43585 was elected leader, sending a full tablet report...
I20260806 22:46:00.684115 5275 ts_tablet_manager.cc:1428] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:00.684475 5276 raft_consensus.cc:348] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.684671 5276 raft_consensus.cc:374] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:00.684581 5277 raft_consensus.cc:348] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.684722 5276 raft_consensus.cc:729] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d510068c7ef456d9694ddb21522cd9e, State: Initialized, Role: FOLLOWER
I20260806 22:46:00.684736 5277 raft_consensus.cc:374] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:00.684769 5277 raft_consensus.cc:729] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b4e6d3855d41a09995fc3673950cd1, State: Initialized, Role: FOLLOWER
I20260806 22:46:00.684840 5276 consensus_queue.cc:260] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.684895 5277 consensus_queue.cc:260] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.685122 5276 ts_tablet_manager.cc:1428] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:00.685181 5277 ts_tablet_manager.cc:1428] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:00.685204 5120 heartbeater.cc:503] Master 127.19.74.254:43585 was elected leader, sending a full tablet report...
I20260806 22:46:00.685256 4979 heartbeater.cc:503] Master 127.19.74.254:43585 was elected leader, sending a full tablet report...
W20260806 22:46:00.739499 5121 tablet.cc:2369] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:00.753790 5282 raft_consensus.cc:482] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:00.753942 5282 raft_consensus.cc:504] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.754360 5282 leader_election.cc:288] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb428ca3f1fa40b29fb483aeecdce8ff (127.19.74.195:34487), 05b4e6d3855d41a09995fc3673950cd1 (127.19.74.194:35119)
I20260806 22:46:00.757153 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.754672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:59397 (local address 127.19.74.195:34487)
0806 22:46:00.754728 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.754730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.754740 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:00.754767 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.754768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.754778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.754845 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:00.754996 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.755453 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.756391 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.756784 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.756872 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.756880 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.757047 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.757085 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:46:00.757155 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.754531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:34487 (local address 127.19.74.193:59397)
0806 22:46:00.754640 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.754651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:00.754695 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:00.754851 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:00.754852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:00.754853 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:00.754927 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.754930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.755531 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.755532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.756296 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.756298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.756871 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.756872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.756971 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.756975 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:00.757041 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:46:00.757081 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":29}
I20260806 22:46:00.757272 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.754663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:52537 (local address 127.19.74.194:35119)
0806 22:46:00.754741 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.754744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.754850 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:46:00.754917 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.754918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.754931 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.754997 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:00.755156 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.755588 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.756486 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.756869 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.756963 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.756971 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.757138 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.757183 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:46:00.757287 5284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.754555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:35119 (local address 127.19.74.193:52537)
0806 22:46:00.754777 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.754786 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:00.754865 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:00.754999 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:00.754999 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:00.755001 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:00.755070 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.755071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.755665 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.755666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.756380 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:00.756382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.756974 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.756975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:00.757051 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.757054 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:00.757112 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:46:00.757152 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":146,"threads_started":1}
I20260806 22:46:00.757504 5211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3bd35071b646baa3e3842f546af713" candidate_uuid: "7d510068c7ef456d9694ddb21522cd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" is_pre_election: true
I20260806 22:46:00.757675 5211 raft_consensus.cc:2454] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d510068c7ef456d9694ddb21522cd9e in term 0.
I20260806 22:46:00.757632 5072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3bd35071b646baa3e3842f546af713" candidate_uuid: "7d510068c7ef456d9694ddb21522cd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b4e6d3855d41a09995fc3673950cd1" is_pre_election: true
I20260806 22:46:00.757798 5072 raft_consensus.cc:2454] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d510068c7ef456d9694ddb21522cd9e in term 0.
I20260806 22:46:00.757936 4862 leader_election.cc:302] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d510068c7ef456d9694ddb21522cd9e, fb428ca3f1fa40b29fb483aeecdce8ff; no voters:
I20260806 22:46:00.758097 5282 raft_consensus.cc:2781] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:00.758162 5282 raft_consensus.cc:482] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:00.758196 5282 raft_consensus.cc:3037] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:00.759083 5282 raft_consensus.cc:504] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.759224 5282 leader_election.cc:288] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [CANDIDATE]: Term 1 election: Requested vote from peers fb428ca3f1fa40b29fb483aeecdce8ff (127.19.74.195:34487), 05b4e6d3855d41a09995fc3673950cd1 (127.19.74.194:35119)
I20260806 22:46:00.759362 5211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3bd35071b646baa3e3842f546af713" candidate_uuid: "7d510068c7ef456d9694ddb21522cd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff"
I20260806 22:46:00.759392 5072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3bd35071b646baa3e3842f546af713" candidate_uuid: "7d510068c7ef456d9694ddb21522cd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b4e6d3855d41a09995fc3673950cd1"
I20260806 22:46:00.759442 5211 raft_consensus.cc:3037] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:00.759473 5072 raft_consensus.cc:3037] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:00.760233 5211 raft_consensus.cc:2454] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d510068c7ef456d9694ddb21522cd9e in term 1.
I20260806 22:46:00.760241 5072 raft_consensus.cc:2454] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d510068c7ef456d9694ddb21522cd9e in term 1.
I20260806 22:46:00.760375 4862 leader_election.cc:302] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d510068c7ef456d9694ddb21522cd9e, fb428ca3f1fa40b29fb483aeecdce8ff; no voters:
I20260806 22:46:00.760488 5282 raft_consensus.cc:2781] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:00.760641 5282 raft_consensus.cc:686] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [term 1 LEADER]: Becoming Leader. State: Replica: 7d510068c7ef456d9694ddb21522cd9e, State: Running, Role: LEADER
I20260806 22:46:00.760763 5282 consensus_queue.cc:237] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } }
I20260806 22:46:00.761636 4780 catalog_manager.cc:5495] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d510068c7ef456d9694ddb21522cd9e (127.19.74.193). New cstate: current_term: 1 leader_uuid: "7d510068c7ef456d9694ddb21522cd9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d510068c7ef456d9694ddb21522cd9e" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 40753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 } health_report { overall_health: UNKNOWN } } }
I20260806 22:46:00.766588 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.764576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.19.74.193:40753)
0806 22:46:00.764610 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.764612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.764635 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:46:00.764693 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.764694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.764709 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.764759 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:00.764863 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.765284 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.765969 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.766156 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.766205 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.766338 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:00.766423 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:00.766481 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.766512 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.766527 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:46:00.768194 5072 raft_consensus.cc:1261] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 1 FOLLOWER]: Refusing update from remote peer 7d510068c7ef456d9694ddb21522cd9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:00.768206 5211 raft_consensus.cc:1261] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 1 FOLLOWER]: Refusing update from remote peer 7d510068c7ef456d9694ddb21522cd9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:00.768507 5285 consensus_queue.cc:1035] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b4e6d3855d41a09995fc3673950cd1" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:00.768642 5282 consensus_queue.cc:1035] T 9d3bd35071b646baa3e3842f546af713 P 7d510068c7ef456d9694ddb21522cd9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb428ca3f1fa40b29fb483aeecdce8ff" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:00.771543 5288 mvcc.cc:205] Tried to move back new op lower bound from 7315686853704159232 to 7315686853676650496. Current Snapshot: MvccSnapshot[applied={T|T < 7315686853704159232}]
I20260806 22:46:00.773702 4780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:00.773780 4780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:00.774516 4780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de1952a3cf3246b5934ccbb81ecf9b13: Sending DeleteTablet for 3 replicas of tablet 9d3bd35071b646baa3e3842f546af713
I20260806 22:46:00.774890 5190 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d3bd35071b646baa3e3842f546af713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:00 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:51420
I20260806 22:46:00.774972 5052 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d3bd35071b646baa3e3842f546af713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:00 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:48306
I20260806 22:46:00.775130 5298 tablet_replica.cc:330] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: stopping tablet replica
I20260806 22:46:00.775175 4910 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d3bd35071b646baa3e3842f546af713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:00 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:51966
I20260806 22:46:00.775225 5298 raft_consensus.cc:2229] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:00.775374 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 4846
I20260806 22:46:00.776830 5298 raft_consensus.cc:2258] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:00.777307 5298 ts_tablet_manager.cc:1905] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:00.778745 5298 ts_tablet_manager.cc:1918] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:00.778827 5298 log.cc:1196] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/9d3bd35071b646baa3e3842f546af713
I20260806 22:46:00.779066 5298 ts_tablet_manager.cc:1939] T 9d3bd35071b646baa3e3842f546af713 P fb428ca3f1fa40b29fb483aeecdce8ff: Deleting consensus metadata
I20260806 22:46:00.779489 4764 catalog_manager.cc:4841] TS fb428ca3f1fa40b29fb483aeecdce8ff (127.19.74.195:34487): tablet 9d3bd35071b646baa3e3842f546af713 (table test-table [id=9a5ff3839e0f4664a9007efe65b7e049]) successfully deleted
I20260806 22:46:00.782850 5299 tablet_replica.cc:330] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1: stopping tablet replica
W20260806 22:46:00.782861 4766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:40753 (error 108)
I20260806 22:46:00.782958 5299 raft_consensus.cc:2229] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:00.783070 5299 raft_consensus.cc:2258] T 9d3bd35071b646baa3e3842f546af713 P 05b4e6d3855d41a09995fc3673950cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:00.783165 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 4984
I20260806 22:46:00.785961 5268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.785788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:40753 (local address 127.0.0.1:51978)
0806 22:46:00.785869 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.785918 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:46:00.786123 4766 catalog_manager.cc:4584] TS 7d510068c7ef456d9694ddb21522cd9e (127.19.74.193:40753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d3bd35071b646baa3e3842f546af713: Network error: Client connection negotiation failed: client connection to 127.19.74.193:40753: connect: Connection refused (error 111)
I20260806 22:46:00.789333 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 5127
I20260806 22:46:00.789367 5268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.789210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:35119 (local address 127.0.0.1:48316)
0806 22:46:00.789297 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:00.789332 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.194:35119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:46:00.794245 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 4747
2026-08-06T22:46:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1974 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T22:46:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:00Z Disabled control of system clock
I20260806 22:46:00.823513 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:35993
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:38879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:35993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:00.905689 5306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:00.905893 5306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:00.905915 5306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:00.907116 5306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:00.907157 5306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:00.907171 5306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:00.907215 5306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:00.907241 5306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:00.908576 5306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:35993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:35993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:00.908840 5306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:00.909142 5306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:00.914403 5313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:00.914386 5315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:00.914387 5312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:00.914539 5306 server_base.cc:1034] running on GCE node
I20260806 22:46:00.914894 5306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:00.915171 5306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:00.916365 5306 hybrid_clock.cc:648] HybridClock initialized: now 1786056360916358 us; error 51 us; skew 500 ppm
I20260806 22:46:00.917624 5306 webserver.cc:460] Webserver started at http://127.19.74.254:36113/ using document root <none> and password file <none>
I20260806 22:46:00.917877 5306 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:00.917948 5306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:00.918098 5306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:00.919198 5306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "cac5bac578c24f26bcd4a10626117f93"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "9d9509795309452f0acaa5eb3300367d"
server_key_iv: "33b4d90980bc05801393cdfbe1ba9235"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.919548 5306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "cac5bac578c24f26bcd4a10626117f93"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "9d9509795309452f0acaa5eb3300367d"
server_key_iv: "33b4d90980bc05801393cdfbe1ba9235"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.920859 5306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:00.921528 5321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:00.921707 5306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:00.921820 5306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "cac5bac578c24f26bcd4a10626117f93"
format_stamp: "Formatted at 2026-08-06 22:46:00 on dist-test-slave-pgkr"
server_key: "9d9509795309452f0acaa5eb3300367d"
server_key_iv: "33b4d90980bc05801393cdfbe1ba9235"
server_key_version: "encryptionkey@0"
I20260806 22:46:00.921890 5306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:00.941416 5306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:00.941723 5306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:00.941867 5306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:00.946134 5306 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:35993
I20260806 22:46:00.946171 5377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:35993 every 8 connection(s)
I20260806 22:46:00.946535 5306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:00.947548 5378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:00.947723 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 5306
I20260806 22:46:00.947834 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:00.948019 19755 external_mini_cluster.cc:1428] Setting key b7bf235379236f0520e08fc1192a1c57
I20260806 22:46:00.950664 5378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93: Bootstrap starting.
I20260806 22:46:00.951333 5378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:00.951645 5378 log.cc:824] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:00.952354 5378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93: No bootstrap required, opened a new log
I20260806 22:46:00.953361 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:00.948540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60166 (local address 127.19.74.254:35993)
0806 22:46:00.948741 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:00.948748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:00.948772 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:46:00.949196 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:00.949202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:00.949258 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:00.949364 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:00.949583 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.950409 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.951481 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:00.951908 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:00.952037 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:00.952758 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:00.952762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:00.952762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:00.952926 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:00.952957 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:00.952967 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:00.953038 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:00.953125 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:46:00.953202 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260806 22:46:00.954416 5378 raft_consensus.cc:348] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5bac578c24f26bcd4a10626117f93" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 35993 } }
I20260806 22:46:00.954566 5378 raft_consensus.cc:374] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:00.954631 5378 raft_consensus.cc:729] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac5bac578c24f26bcd4a10626117f93, State: Initialized, Role: FOLLOWER
I20260806 22:46:00.954774 5378 consensus_queue.cc:260] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5bac578c24f26bcd4a10626117f93" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 35993 } }
I20260806 22:46:00.954866 5378 raft_consensus.cc:388] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:00.954910 5378 raft_consensus.cc:482] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:00.954957 5378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:00.955552 5378 raft_consensus.cc:504] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5bac578c24f26bcd4a10626117f93" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 35993 } }
I20260806 22:46:00.955678 5378 leader_election.cc:302] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cac5bac578c24f26bcd4a10626117f93; no voters:
I20260806 22:46:00.955853 5378 leader_election.cc:288] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:00.955899 5383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:00.956063 5383 raft_consensus.cc:686] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [term 1 LEADER]: Becoming Leader. State: Replica: cac5bac578c24f26bcd4a10626117f93, State: Running, Role: LEADER
I20260806 22:46:00.956122 5378 sys_catalog.cc:564] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:00.956193 5383 consensus_queue.cc:237] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5bac578c24f26bcd4a10626117f93" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 35993 } }
I20260806 22:46:00.956554 5385 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cac5bac578c24f26bcd4a10626117f93. Latest consensus state: current_term: 1 leader_uuid: "cac5bac578c24f26bcd4a10626117f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5bac578c24f26bcd4a10626117f93" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 35993 } } }
I20260806 22:46:00.956586 5384 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cac5bac578c24f26bcd4a10626117f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5bac578c24f26bcd4a10626117f93" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 35993 } } }
I20260806 22:46:00.956835 5384 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:00.956820 5385 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:00.957118 5392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:00.957602 5392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:00.959182 5392 catalog_manager.cc:1347] Generated new cluster ID: 940e952d160c4c9fad5df9d7babd8aac
I20260806 22:46:00.959233 5392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:00.971586 5392 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:00.972064 5392 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:00.976104 5392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93: Generated new TSK 0
I20260806 22:46:00.976277 5392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:00.984516 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:35993
--builtin_ntp_servers=127.19.74.212:38879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:01.087248 5402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:01.087457 5402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:01.087477 5402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:01.088574 5402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:01.088637 5402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:01.088661 5402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:46:01.089993 5402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:35993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:01.090227 5402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:01.090549 5402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:01.095649 5407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:01.095767 5402 server_base.cc:1034] running on GCE node
W20260806 22:46:01.095830 5408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:01.095630 5410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:01.096168 5402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:01.096462 5402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:01.097628 5402 hybrid_clock.cc:648] HybridClock initialized: now 1786056361097614 us; error 31 us; skew 500 ppm
I20260806 22:46:01.098922 5402 webserver.cc:460] Webserver started at http://127.19.74.193:33053/ using document root <none> and password file <none>
I20260806 22:46:01.099169 5402 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:01.099237 5402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:01.099386 5402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:01.100402 5402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "4723738da8124754b14bcabc63314cea"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "6481de4029be6f478e47351cfc91689d"
server_key_iv: "d4a780c55f924e39a72997b00aa393c6"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.100805 5402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "4723738da8124754b14bcabc63314cea"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "6481de4029be6f478e47351cfc91689d"
server_key_iv: "d4a780c55f924e39a72997b00aa393c6"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.102213 5402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:01.103039 5416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.103219 5402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:01.103322 5402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "4723738da8124754b14bcabc63314cea"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "6481de4029be6f478e47351cfc91689d"
server_key_iv: "d4a780c55f924e39a72997b00aa393c6"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.103397 5402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:01.111315 5402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:01.111593 5402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:01.111742 5402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:01.111941 5402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:01.112334 5402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:01.112397 5402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.112439 5402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:01.112466 5402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.118273 5402 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:38977
I20260806 22:46:01.118315 5534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:38977 every 8 connection(s)
I20260806 22:46:01.118665 5402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:01.120906 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 5402
I20260806 22:46:01.121019 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:01.121243 19755 external_mini_cluster.cc:1428] Setting key 4eabf46a0394456da46d1f36d6bb42b7
I20260806 22:46:01.122143 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:35993
--builtin_ntp_servers=127.19.74.212:38879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:46:01.124347 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.119310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:48541 (local address 127.19.74.254:35993)
0806 22:46:01.119384 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.119386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.119623 (+ 237us) server_negotiation.cc:407] Connection header received
0806 22:46:01.120115 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.120116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.120129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.120177 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:01.120844 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.121517 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.122851 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.123056 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.123143 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.124027 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:01.124028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:01.124028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:01.124033 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:01.124050 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:01.124055 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:01.124114 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.124216 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.124242 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:46:01.124504 5537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.119294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:35993 (local address 127.19.74.193:48541)
0806 22:46:01.119483 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.119541 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:01.120027 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:01.120246 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:01.120249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:01.120257 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:01.120727 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.120731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.121794 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.121795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.122768 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.122770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.123156 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.123158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.123290 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.123854 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:01.123855 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:01.123882 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:01.123927 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:01.124131 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:01.124136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:01.124138 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:46:01.124189 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:46:01.124267 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260806 22:46:01.124974 5535 heartbeater.cc:347] Connected to a master server at 127.19.74.254:35993
I20260806 22:46:01.125064 5535 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:01.125239 5535 heartbeater.cc:511] Master 127.19.74.254:35993 requested a full tablet report, sending...
I20260806 22:46:01.125764 5339 ts_manager.cc:194] Registered new tserver with Master: 4723738da8124754b14bcabc63314cea (127.19.74.193:38977)
I20260806 22:46:01.126748 5339 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.193:48541
W20260806 22:46:01.204816 5538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:01.205014 5538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:01.205036 5538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:01.206187 5538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:01.206224 5538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:01.206248 5538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:46:01.207614 5538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:35993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:01.207867 5538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:01.208164 5538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:01.213084 5546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:01.213078 5544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:01.213079 5543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:01.213357 5538 server_base.cc:1034] running on GCE node
I20260806 22:46:01.213583 5538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:01.213841 5538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:01.214983 5538 hybrid_clock.cc:648] HybridClock initialized: now 1786056361214968 us; error 26 us; skew 500 ppm
I20260806 22:46:01.216219 5538 webserver.cc:460] Webserver started at http://127.19.74.194:35763/ using document root <none> and password file <none>
I20260806 22:46:01.216430 5538 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:01.216470 5538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:01.216559 5538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:01.217557 5538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "432f7723b3d44f22868e18135df43549"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "2c580e9c2d6965ded09a79dae4624093"
server_key_iv: "f99e9b9a32f5ae654182788162457af6"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.217945 5538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "432f7723b3d44f22868e18135df43549"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "2c580e9c2d6965ded09a79dae4624093"
server_key_iv: "f99e9b9a32f5ae654182788162457af6"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.219256 5538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:01.219961 5552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.220131 5538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:01.220239 5538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "432f7723b3d44f22868e18135df43549"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "2c580e9c2d6965ded09a79dae4624093"
server_key_iv: "f99e9b9a32f5ae654182788162457af6"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.220312 5538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:01.229882 5538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:01.230173 5538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:01.230322 5538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:01.230543 5538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:01.230902 5538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:01.230958 5538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.231001 5538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:01.231029 5538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.236729 5538 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:44043
I20260806 22:46:01.236796 5671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:44043 every 8 connection(s)
I20260806 22:46:01.237154 5538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:01.238154 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 5538
I20260806 22:46:01.238255 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:01.238448 19755 external_mini_cluster.cc:1428] Setting key 067224b607434ff4fab053f0ce486ab9
I20260806 22:46:01.239378 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:35993
--builtin_ntp_servers=127.19.74.212:38879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:46:01.242466 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.237906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:43753 (local address 127.19.74.254:35993)
0806 22:46:01.237977 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.237979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.238193 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:46:01.238823 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.238825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.238836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.238879 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:01.239585 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.240058 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.241055 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.241258 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.241315 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.242206 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:01.242207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:01.242207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:01.242212 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:01.242227 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:01.242231 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:01.242276 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.242389 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.242411 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:46:01.242771 5674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.237870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:35993 (local address 127.19.74.194:43753)
0806 22:46:01.238052 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.238098 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:01.238716 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:01.238944 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:01.238948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:01.238958 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:01.239478 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.239483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.240153 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.240154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.240983 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.240986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.241349 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.241350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.241503 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.242041 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:01.242042 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:01.242083 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:01.242120 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:01.242304 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:01.242311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:01.242314 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:01.242384 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:46:01.242490 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260806 22:46:01.243255 5672 heartbeater.cc:347] Connected to a master server at 127.19.74.254:35993
I20260806 22:46:01.243347 5672 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:01.243525 5672 heartbeater.cc:511] Master 127.19.74.254:35993 requested a full tablet report, sending...
I20260806 22:46:01.243901 5339 ts_manager.cc:194] Registered new tserver with Master: 432f7723b3d44f22868e18135df43549 (127.19.74.194:44043)
I20260806 22:46:01.244262 5339 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.194:43753
W20260806 22:46:01.327159 5675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:01.327381 5675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:01.327415 5675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:01.328642 5675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:01.328699 5675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:01.328739 5675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:46:01.330080 5675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:38879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:35993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:01.330367 5675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:01.330729 5675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:01.335672 5684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:01.335721 5681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:01.335912 5675 server_base.cc:1034] running on GCE node
W20260806 22:46:01.335955 5682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:01.336176 5675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:01.336421 5675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:01.337579 5675 hybrid_clock.cc:648] HybridClock initialized: now 1786056361337568 us; error 42 us; skew 500 ppm
I20260806 22:46:01.338807 5675 webserver.cc:460] Webserver started at http://127.19.74.195:32853/ using document root <none> and password file <none>
I20260806 22:46:01.339057 5675 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:01.339128 5675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:01.339274 5675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:01.340307 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "81538c9ce066497f8b525957e3bdf810"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "f0af798cfba959fa9ed7687c8f3607ed"
server_key_iv: "8f1cc4424e5c9e590dc7f2b7da50976f"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.340682 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "81538c9ce066497f8b525957e3bdf810"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "f0af798cfba959fa9ed7687c8f3607ed"
server_key_iv: "8f1cc4424e5c9e590dc7f2b7da50976f"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.341936 5675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:01.342604 5690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.342774 5675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:01.342869 5675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "81538c9ce066497f8b525957e3bdf810"
format_stamp: "Formatted at 2026-08-06 22:46:01 on dist-test-slave-pgkr"
server_key: "f0af798cfba959fa9ed7687c8f3607ed"
server_key_iv: "8f1cc4424e5c9e590dc7f2b7da50976f"
server_key_version: "encryptionkey@0"
I20260806 22:46:01.342949 5675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:01.354346 5675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:01.354601 5675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:01.354791 5675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:01.354995 5675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:01.355343 5675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:01.355388 5675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.355427 5675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:01.355454 5675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:01.361348 5675 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:36051
I20260806 22:46:01.361389 5806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:36051 every 8 connection(s)
I20260806 22:46:01.361688 5675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:01.365636 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 5675
I20260806 22:46:01.365744 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:01.365963 19755 external_mini_cluster.cc:1428] Setting key da8553a6d18373d0b4fd4256a51c2dc7
I20260806 22:46:01.366669 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.362344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:34157 (local address 127.19.74.254:35993)
0806 22:46:01.362430 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.362432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.362741 (+ 309us) server_negotiation.cc:407] Connection header received
0806 22:46:01.363166 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.363167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.363177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.363227 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:01.363815 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.364259 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.365154 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.365322 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.365369 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.366346 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:01.366347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:01.366348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:01.366353 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:01.366368 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:01.366373 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:01.366427 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.366530 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.366558 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:46:01.366853 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.362329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:35993 (local address 127.19.74.195:34157)
0806 22:46:01.362555 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.362589 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:01.363080 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:01.363277 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:01.363280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:01.363288 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:01.363754 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.363758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.364331 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.364332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.365076 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.365078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.365391 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.365392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.365562 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.366170 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:01.366170 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:01.366197 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:01.366219 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:01.366441 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:01.366447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:01.366450 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:01.366518 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:46:01.366636 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
I20260806 22:46:01.367365 5807 heartbeater.cc:347] Connected to a master server at 127.19.74.254:35993
I20260806 22:46:01.367479 5807 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:01.367658 5807 heartbeater.cc:511] Master 127.19.74.254:35993 requested a full tablet report, sending...
I20260806 22:46:01.368026 5339 ts_manager.cc:194] Registered new tserver with Master: 81538c9ce066497f8b525957e3bdf810 (127.19.74.195:36051)
I20260806 22:46:01.368343 5339 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.195:34157
I20260806 22:46:01.377925 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:46:01.381762 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.379364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60180 (local address 127.19.74.254:35993)
0806 22:46:01.379439 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.379441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.379496 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:46:01.379534 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.379535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.379542 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.379582 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:01.379702 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.380134 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.380831 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.380998 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.381044 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.381557 (+ 513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:01.381558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:01.381558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:01.381562 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:01.381573 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:01.381576 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:01.381616 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.381648 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.381666 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:46:01.383186 5339 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:01.384784 5339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:01.391168 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.387013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:38977 (local address 127.0.0.1:50812)
0806 22:46:01.387214 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.387227 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:01.387291 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:01.387685 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:01.387689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:01.387690 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:01.387749 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.387752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.389133 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.389134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.390030 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.390033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.390860 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.390861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.390935 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.390949 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:46:01.391028 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:46:01.391080 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":4,"spinlock_wait_cycles":768,"thread_start_us":102,"threads_started":1}
I20260806 22:46:01.391260 5820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.387447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.19.74.194:44043)
0806 22:46:01.387624 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.387628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.387636 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:01.387672 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.387674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.387687 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.387822 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:01.388021 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.388696 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.390102 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.390863 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.390936 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.391038 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.391134 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.391176 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260806 22:46:01.391391 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.387013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:36051 (local address 127.0.0.1:33796)
0806 22:46:01.387182 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.387226 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:01.387289 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:01.387806 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:01.387807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:01.387809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:01.387852 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.387854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.389103 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.389104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.390206 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.390209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.391162 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.391164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.391221 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.391224 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:01.391279 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:46:01.391326 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260806 22:46:01.391451 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.387146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33796 (local address 127.19.74.195:36051)
0806 22:46:01.387589 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.387603 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.387613 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:01.387651 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.387653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.387668 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.387744 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:01.387913 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.389004 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.390282 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.390975 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.391046 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.391126 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.391314 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.391365 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":77,"threads_started":1}
I20260806 22:46:01.391795 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.387243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:44043 (local address 127.0.0.1:47414)
0806 22:46:01.387398 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.387408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:01.387443 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:01.387886 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:01.387886 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:01.387888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:01.387922 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.387923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.388786 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.388787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.390007 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.390009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.391014 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.391016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.391066 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.391069 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:01.391120 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:46:01.391152 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":63,"threads_started":1}
I20260806 22:46:01.392040 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.387317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50812 (local address 127.19.74.193:38977)
0806 22:46:01.387525 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.387530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.387541 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:01.387581 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.387584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.387600 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.387679 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:01.387836 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.389029 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.390135 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.390772 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.390860 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.390952 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.391900 (+ 948us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.391952 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260806 22:46:01.392213 5739 tablet_service.cc:1467] Processing CreateTablet for tablet f69f5494b7894a8487c712d157421076 (DEFAULT_TABLE table=test-table [id=4ebea9a267154bea986c388fe85ef38a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:01.392639 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69f5494b7894a8487c712d157421076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:01.392532 5602 tablet_service.cc:1467] Processing CreateTablet for tablet f69f5494b7894a8487c712d157421076 (DEFAULT_TABLE table=test-table [id=4ebea9a267154bea986c388fe85ef38a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:01.392647 5467 tablet_service.cc:1467] Processing CreateTablet for tablet f69f5494b7894a8487c712d157421076 (DEFAULT_TABLE table=test-table [id=4ebea9a267154bea986c388fe85ef38a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:01.392895 5602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69f5494b7894a8487c712d157421076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:01.392932 5467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69f5494b7894a8487c712d157421076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:01.395313 5821 tablet_bootstrap.cc:492] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: Bootstrap starting.
I20260806 22:46:01.395931 5821 tablet_bootstrap.cc:654] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:01.396189 5822 tablet_bootstrap.cc:492] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea: Bootstrap starting.
I20260806 22:46:01.396292 5821 log.cc:824] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:01.396341 5824 tablet_bootstrap.cc:492] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Bootstrap starting.
I20260806 22:46:01.396900 5822 tablet_bootstrap.cc:654] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:01.396997 5824 tablet_bootstrap.cc:654] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:01.397145 5821 tablet_bootstrap.cc:492] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: No bootstrap required, opened a new log
I20260806 22:46:01.397222 5821 ts_tablet_manager.cc:1397] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:01.397253 5822 log.cc:824] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:01.397332 5824 log.cc:824] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:01.397980 5824 tablet_bootstrap.cc:492] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: No bootstrap required, opened a new log
I20260806 22:46:01.397990 5822 tablet_bootstrap.cc:492] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea: No bootstrap required, opened a new log
I20260806 22:46:01.398054 5824 ts_tablet_manager.cc:1397] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:01.398116 5822 ts_tablet_manager.cc:1397] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:01.399362 5821 raft_consensus.cc:348] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.399510 5821 raft_consensus.cc:374] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:01.399554 5821 raft_consensus.cc:729] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81538c9ce066497f8b525957e3bdf810, State: Initialized, Role: FOLLOWER
I20260806 22:46:01.399663 5821 consensus_queue.cc:260] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.399995 5821 ts_tablet_manager.cc:1428] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:01.400004 5807 heartbeater.cc:503] Master 127.19.74.254:35993 was elected leader, sending a full tablet report...
I20260806 22:46:01.400662 5822 raft_consensus.cc:348] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.400643 5824 raft_consensus.cc:348] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.400801 5824 raft_consensus.cc:374] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:01.400801 5822 raft_consensus.cc:374] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:01.400835 5824 raft_consensus.cc:729] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432f7723b3d44f22868e18135df43549, State: Initialized, Role: FOLLOWER
I20260806 22:46:01.400835 5822 raft_consensus.cc:729] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4723738da8124754b14bcabc63314cea, State: Initialized, Role: FOLLOWER
I20260806 22:46:01.400966 5824 consensus_queue.cc:260] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.400966 5822 consensus_queue.cc:260] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.401217 5822 ts_tablet_manager.cc:1428] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:46:01.401217 5824 ts_tablet_manager.cc:1428] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:46:01.401328 5672 heartbeater.cc:503] Master 127.19.74.254:35993 was elected leader, sending a full tablet report...
I20260806 22:46:01.401360 5535 heartbeater.cc:503] Master 127.19.74.254:35993 was elected leader, sending a full tablet report...
I20260806 22:46:01.463438 5831 raft_consensus.cc:482] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:01.463587 5831 raft_consensus.cc:504] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.463997 5831 leader_election.cc:288] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4723738da8124754b14bcabc63314cea (127.19.74.193:38977), 81538c9ce066497f8b525957e3bdf810 (127.19.74.195:36051)
I20260806 22:46:01.467748 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.464279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:47301 (local address 127.19.74.195:36051)
0806 22:46:01.464334 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.464335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.464466 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:46:01.464503 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.464504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.464511 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.464571 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:01.464710 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.465374 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.466829 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.467389 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.467467 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.467476 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.467637 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.467679 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:46:01.467748 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.464249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:56975 (local address 127.19.74.193:38977)
0806 22:46:01.464294 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.464296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.464306 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:01.464340 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.464341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.464352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.464410 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:01.464585 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.465326 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.466829 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.467389 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.467467 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.467476 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.467643 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.467683 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:46:01.467760 5674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.464176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:38977 (local address 127.19.74.194:56975)
0806 22:46:01.464226 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.464238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:01.464280 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:01.464432 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:01.464433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:01.464435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:01.464511 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.464513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.465397 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.465397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.466739 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.466743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.467473 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.467474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.467552 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.467556 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:01.467624 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:46:01.467653 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":61}
I20260806 22:46:01.467773 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.464178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:36051 (local address 127.19.74.194:47301)
0806 22:46:01.464379 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.464386 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:01.464448 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:01.464585 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:01.464586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:01.464588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:01.464645 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.464646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.465440 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.465441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.466739 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:01.466743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.467474 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.467475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:01.467552 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.467557 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:01.467612 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:46:01.467653 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":48,"thread_start_us":96,"threads_started":1}
I20260806 22:46:01.468168 5487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69f5494b7894a8487c712d157421076" candidate_uuid: "432f7723b3d44f22868e18135df43549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4723738da8124754b14bcabc63314cea" is_pre_election: true
I20260806 22:46:01.468346 5487 raft_consensus.cc:2454] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 432f7723b3d44f22868e18135df43549 in term 0.
I20260806 22:46:01.468420 5760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69f5494b7894a8487c712d157421076" candidate_uuid: "432f7723b3d44f22868e18135df43549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81538c9ce066497f8b525957e3bdf810" is_pre_election: true
I20260806 22:46:01.468575 5760 raft_consensus.cc:2454] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 432f7723b3d44f22868e18135df43549 in term 0.
I20260806 22:46:01.468628 5555 leader_election.cc:302] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 432f7723b3d44f22868e18135df43549, 4723738da8124754b14bcabc63314cea; no voters:
I20260806 22:46:01.468760 5831 raft_consensus.cc:2781] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:01.468812 5831 raft_consensus.cc:482] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:01.468829 5831 raft_consensus.cc:3037] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:01.469496 5831 raft_consensus.cc:504] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.469635 5831 leader_election.cc:288] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [CANDIDATE]: Term 1 election: Requested vote from peers 4723738da8124754b14bcabc63314cea (127.19.74.193:38977), 81538c9ce066497f8b525957e3bdf810 (127.19.74.195:36051)
I20260806 22:46:01.469765 5487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69f5494b7894a8487c712d157421076" candidate_uuid: "432f7723b3d44f22868e18135df43549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4723738da8124754b14bcabc63314cea"
I20260806 22:46:01.469789 5760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69f5494b7894a8487c712d157421076" candidate_uuid: "432f7723b3d44f22868e18135df43549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81538c9ce066497f8b525957e3bdf810"
I20260806 22:46:01.469852 5760 raft_consensus.cc:3037] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:01.469847 5487 raft_consensus.cc:3037] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:01.470768 5760 raft_consensus.cc:2454] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 432f7723b3d44f22868e18135df43549 in term 1.
I20260806 22:46:01.470768 5487 raft_consensus.cc:2454] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 432f7723b3d44f22868e18135df43549 in term 1.
I20260806 22:46:01.470929 5555 leader_election.cc:302] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 432f7723b3d44f22868e18135df43549, 4723738da8124754b14bcabc63314cea; no voters:
I20260806 22:46:01.471026 5831 raft_consensus.cc:2781] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:01.471156 5831 raft_consensus.cc:686] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 1 LEADER]: Becoming Leader. State: Replica: 432f7723b3d44f22868e18135df43549, State: Running, Role: LEADER
I20260806 22:46:01.471249 5831 consensus_queue.cc:237] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } }
I20260806 22:46:01.472100 5338 catalog_manager.cc:5495] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 reported cstate change: term changed from 0 to 1, leader changed from <none> to 432f7723b3d44f22868e18135df43549 (127.19.74.194). New cstate: current_term: 1 leader_uuid: "432f7723b3d44f22868e18135df43549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "432f7723b3d44f22868e18135df43549" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 44043 } health_report { overall_health: HEALTHY } } }
I20260806 22:46:01.484277 5820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.482341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.19.74.194:44043)
0806 22:46:01.482388 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:01.482389 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:01.482398 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:01.482427 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:01.482427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:01.482442 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:01.482500 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:01.482642 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.483064 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.483690 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:01.483849 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:01.483894 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:01.484005 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:01.484107 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:01.484169 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:01.484199 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:46:01.484214 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:46:01.485915 5487 raft_consensus.cc:1261] T f69f5494b7894a8487c712d157421076 P 4723738da8124754b14bcabc63314cea [term 1 FOLLOWER]: Refusing update from remote peer 432f7723b3d44f22868e18135df43549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:01.485960 5760 raft_consensus.cc:1261] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 1 FOLLOWER]: Refusing update from remote peer 432f7723b3d44f22868e18135df43549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:01.486318 5831 consensus_queue.cc:1035] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81538c9ce066497f8b525957e3bdf810" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 36051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:01.486456 5834 consensus_queue.cc:1035] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4723738da8124754b14bcabc63314cea" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:46:01.487632 5673 tablet.cc:2369] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:01.488986 5838 mvcc.cc:205] Tried to move back new op lower bound from 7315686856643973120 to 7315686856586924032. Current Snapshot: MvccSnapshot[applied={T|T < 7315686856643973120}]
I20260806 22:46:01.489351 5839 mvcc.cc:205] Tried to move back new op lower bound from 7315686856643973120 to 7315686856586924032. Current Snapshot: MvccSnapshot[applied={T|T < 7315686856643973120}]
I20260806 22:46:01.490535 5840 mvcc.cc:205] Tried to move back new op lower bound from 7315686856643973120 to 7315686856586924032. Current Snapshot: MvccSnapshot[applied={T|T < 7315686856643973120}]
I20260806 22:46:01.491287 5339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:01.491377 5339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:01.492043 5339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cac5bac578c24f26bcd4a10626117f93: Sending DeleteTablet for 3 replicas of tablet f69f5494b7894a8487c712d157421076
I20260806 22:46:01.492409 5602 tablet_service.cc:1514] Processing DeleteTablet for tablet f69f5494b7894a8487c712d157421076 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:01 UTC) from {username='slave'} at 127.0.0.1:47414
I20260806 22:46:01.492473 5739 tablet_service.cc:1514] Processing DeleteTablet for tablet f69f5494b7894a8487c712d157421076 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:01 UTC) from {username='slave'} at 127.0.0.1:33796
I20260806 22:46:01.492491 5467 tablet_service.cc:1514] Processing DeleteTablet for tablet f69f5494b7894a8487c712d157421076 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:01 UTC) from {username='slave'} at 127.0.0.1:50812
I20260806 22:46:01.492718 5848 tablet_replica.cc:330] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: stopping tablet replica
I20260806 22:46:01.492808 5848 raft_consensus.cc:2229] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:01.492869 5848 raft_consensus.cc:2258] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:01.492962 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 5402
I20260806 22:46:01.494467 5848 ts_tablet_manager.cc:1905] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:01.496119 5848 ts_tablet_manager.cc:1918] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:01.496173 5848 log.cc:1196] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/f69f5494b7894a8487c712d157421076
I20260806 22:46:01.496408 5848 ts_tablet_manager.cc:1939] T f69f5494b7894a8487c712d157421076 P 432f7723b3d44f22868e18135df43549: Deleting consensus metadata
I20260806 22:46:01.496726 5323 catalog_manager.cc:4841] TS 432f7723b3d44f22868e18135df43549 (127.19.74.194:44043): tablet f69f5494b7894a8487c712d157421076 (table test-table [id=4ebea9a267154bea986c388fe85ef38a]) successfully deleted
I20260806 22:46:01.498732 5849 tablet_replica.cc:330] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: stopping tablet replica
I20260806 22:46:01.498853 5849 raft_consensus.cc:2229] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:01.498970 5849 raft_consensus.cc:2258] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:01.499372 5849 ts_tablet_manager.cc:1905] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:01.500782 5849 ts_tablet_manager.cc:1918] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:01.500866 5849 log.cc:1196] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/f69f5494b7894a8487c712d157421076
I20260806 22:46:01.501147 5849 ts_tablet_manager.cc:1939] T f69f5494b7894a8487c712d157421076 P 81538c9ce066497f8b525957e3bdf810: Deleting consensus metadata
I20260806 22:46:01.501530 5323 catalog_manager.cc:4841] TS 81538c9ce066497f8b525957e3bdf810 (127.19.74.195:36051): tablet f69f5494b7894a8487c712d157421076 (table test-table [id=4ebea9a267154bea986c388fe85ef38a]) successfully deleted
W20260806 22:46:01.503727 5324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:38977 (error 108)
I20260806 22:46:01.503930 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 5538
I20260806 22:46:01.504848 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:01.504091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:38977 (local address 127.0.0.1:50816)
0806 22:46:01.504763 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:01.504809 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:38977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":651}
W20260806 22:46:01.507158 5324 catalog_manager.cc:4584] TS 4723738da8124754b14bcabc63314cea (127.19.74.193:38977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f69f5494b7894a8487c712d157421076: Network error: Client connection negotiation failed: client connection to 127.19.74.193:38977: connect: Connection refused (error 111)
I20260806 22:46:01.511164 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 5675
I20260806 22:46:01.516539 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 5306
2026-08-06T22:46:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (721 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:01 dist-test-slave-pgkr krb5kdc[5854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:01 dist-test-slave-pgkr krb5kdc[5854](info): set up 2 sockets
Aug 06 22:46:01 dist-test-slave-pgkr krb5kdc[5854](info): commencing operation
krb5kdc: starting...
W20260806 22:46:02.564555 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:02 dist-test-slave-pgkr krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:02.684527 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:40845
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:40845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:02.765547 5867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:02.765779 5867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:02.765831 5867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:02.767300 5867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:02.767472 5867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:02.767503 5867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:02.767532 5867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:02.767566 5867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:02.769229 5867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:40845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:40845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:02.769519 5867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:02.769762 5867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:02.774573 5872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:02.774578 5875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:02.774613 5873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:02.775141 5867 server_base.cc:1034] running on GCE node
I20260806 22:46:02.775369 5867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:02.775640 5867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:02.776803 5867 hybrid_clock.cc:648] HybridClock initialized: now 1786056362776784 us; error 29 us; skew 500 ppm
Aug 06 22:46:02 dist-test-slave-pgkr krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056362, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:02.779281 5867 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:02.779651 5867 webserver.cc:460] Webserver started at http://127.19.74.254:34561/ using document root <none> and password file <none>
I20260806 22:46:02.779831 5867 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:02.779870 5867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:02.779953 5867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:02.780879 5867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "37fc77ec3e134d349cf54b9178cc9182"
format_stamp: "Formatted at 2026-08-06 22:46:02 on dist-test-slave-pgkr"
server_key: "465bc846285ec094775c84701973b2d0"
server_key_iv: "157deae0cae7f672a58b96a3aca6d596"
server_key_version: "encryptionkey@0"
I20260806 22:46:02.781185 5867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "37fc77ec3e134d349cf54b9178cc9182"
format_stamp: "Formatted at 2026-08-06 22:46:02 on dist-test-slave-pgkr"
server_key: "465bc846285ec094775c84701973b2d0"
server_key_iv: "157deae0cae7f672a58b96a3aca6d596"
server_key_version: "encryptionkey@0"
I20260806 22:46:02.782368 5867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:02.783105 5882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:02.783324 5867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:02.783433 5867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "37fc77ec3e134d349cf54b9178cc9182"
format_stamp: "Formatted at 2026-08-06 22:46:02 on dist-test-slave-pgkr"
server_key: "465bc846285ec094775c84701973b2d0"
server_key_iv: "157deae0cae7f672a58b96a3aca6d596"
server_key_version: "encryptionkey@0"
I20260806 22:46:02.783511 5867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:02.809943 5867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:02.812435 5867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:02.812592 5867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:02.816926 5867 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:40845
I20260806 22:46:02.816980 5938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:40845 every 8 connection(s)
I20260806 22:46:02.817373 5867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:02.818356 5939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:02.819622 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 5867
I20260806 22:46:02.819726 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:02.819975 19755 external_mini_cluster.cc:1428] Setting key 6c71e26c0274eabe5d76ae5a335998fa
I20260806 22:46:02.821048 5939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182: Bootstrap starting.
I20260806 22:46:02.821741 5939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:02.822129 5939 log.cc:824] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:02.822963 5939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182: No bootstrap required, opened a new log
I20260806 22:46:02.825305 5939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37fc77ec3e134d349cf54b9178cc9182" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 40845 } }
I20260806 22:46:02.825487 5939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:02.825518 5939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fc77ec3e134d349cf54b9178cc9182, State: Initialized, Role: FOLLOWER
I20260806 22:46:02.825665 5939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37fc77ec3e134d349cf54b9178cc9182" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 40845 } }
I20260806 22:46:02.825796 5939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:02.825852 5939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:02.825903 5939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:02.826710 5939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37fc77ec3e134d349cf54b9178cc9182" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 40845 } }
I20260806 22:46:02.826826 5939 leader_election.cc:302] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37fc77ec3e134d349cf54b9178cc9182; no voters:
I20260806 22:46:02.827023 5939 leader_election.cc:288] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:02.827059 5944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:02.827168 5944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [term 1 LEADER]: Becoming Leader. State: Replica: 37fc77ec3e134d349cf54b9178cc9182, State: Running, Role: LEADER
I20260806 22:46:02.827317 5939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:02.827322 5944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37fc77ec3e134d349cf54b9178cc9182" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 40845 } }
Aug 06 22:46:02 dist-test-slave-pgkr krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:02.827800 5945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37fc77ec3e134d349cf54b9178cc9182. Latest consensus state: current_term: 1 leader_uuid: "37fc77ec3e134d349cf54b9178cc9182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37fc77ec3e134d349cf54b9178cc9182" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 40845 } } }
I20260806 22:46:02.827905 5945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:02.828131 5946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37fc77ec3e134d349cf54b9178cc9182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37fc77ec3e134d349cf54b9178cc9182" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 40845 } } }
I20260806 22:46:02.828439 5946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:02.828637 5951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:02.829237 5951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:02.830988 5951 catalog_manager.cc:1347] Generated new cluster ID: b8877306734748c49044cca79b358a8c
I20260806 22:46:02.831064 5951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:02.832545 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:02.820624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37242 (local address 127.19.74.254:40845)
0806 22:46:02.820818 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:02.820834 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:02.820861 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:46:02.821309 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:02.821316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:02.821366 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:02.821492 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:02.822364 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:02.823514 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:02.824685 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:02.825098 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:02.825196 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:02.828172 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:02.828177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:02.828177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:02.828190 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:02.830691 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:02.831250 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:02.831253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:02.831253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:02.831361 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:02.831599 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:02.831600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:02.831600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:02.831881 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:02.831998 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:02.832180 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 22:46:02.832321 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260806 22:46:02.837896 5951 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:02.838411 5951 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:02.859927 5951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182: Generated new TSK 0
I20260806 22:46:02.860162 5951 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:46:02.897976 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:40845
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:02.981082 5965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:02.981276 5965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:02.981297 5965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:02.982483 5965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:02.982527 5965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:02.982553 5965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:46:02.984026 5965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:40845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:02.984292 5965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:02.984535 5965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:02.989727 5970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:02.989769 5971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:02.989764 5973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:02.989825 5965 server_base.cc:1034] running on GCE node
I20260806 22:46:02.990168 5965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:02.990406 5965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:02.991565 5965 hybrid_clock.cc:648] HybridClock initialized: now 1786056362991562 us; error 37 us; skew 500 ppm
Aug 06 22:46:02 dist-test-slave-pgkr krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056362, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:02.994035 5965 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:46:02.994496 5965 webserver.cc:460] Webserver started at http://127.19.74.193:36361/ using document root <none> and password file <none>
I20260806 22:46:02.994793 5965 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:02.994864 5965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:02.995018 5965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:02.995996 5965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "0c9ca5f270e7489296e34ffd140254e4"
format_stamp: "Formatted at 2026-08-06 22:46:02 on dist-test-slave-pgkr"
server_key: "d2e2672a6e8ce3004d23ee7beca90276"
server_key_iv: "c557f2e4876baeb31cf82c83525c559f"
server_key_version: "encryptionkey@0"
I20260806 22:46:02.996393 5965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "0c9ca5f270e7489296e34ffd140254e4"
format_stamp: "Formatted at 2026-08-06 22:46:02 on dist-test-slave-pgkr"
server_key: "d2e2672a6e8ce3004d23ee7beca90276"
server_key_iv: "c557f2e4876baeb31cf82c83525c559f"
server_key_version: "encryptionkey@0"
I20260806 22:46:02.997709 5965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:02.998416 5980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:02.998588 5965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:02.998728 5965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "0c9ca5f270e7489296e34ffd140254e4"
format_stamp: "Formatted at 2026-08-06 22:46:02 on dist-test-slave-pgkr"
server_key: "d2e2672a6e8ce3004d23ee7beca90276"
server_key_iv: "c557f2e4876baeb31cf82c83525c559f"
server_key_version: "encryptionkey@0"
I20260806 22:46:02.998811 5965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:03.008469 5965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:03.010254 5965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:03.010421 5965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:03.010722 5965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:03.011055 5965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:03.011114 5965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.011157 5965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:03.011184 5965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.017373 5965 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:39121
I20260806 22:46:03.017423 6097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:39121 every 8 connection(s)
I20260806 22:46:03.017755 5965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:03.023610 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 5965
I20260806 22:46:03.023746 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
Aug 06 22:46:03 dist-test-slave-pgkr krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056362, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:03.024010 19755 external_mini_cluster.cc:1428] Setting key f8c84d0044a6c92a6709c451c683285c
I20260806 22:46:03.027633 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.018468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:41903 (local address 127.19.74.254:40845)
0806 22:46:03.018547 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.018549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.018847 (+ 298us) server_negotiation.cc:407] Connection header received
0806 22:46:03.019320 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.019322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.019332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.019385 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:03.020295 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.020799 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.021754 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.021945 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.021997 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.024454 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:03.024455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:03.024455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:03.024464 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:03.026188 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:03.026571 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:03.026572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:03.026572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:03.026663 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:03.026973 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:03.026974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:03.026974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:03.027081 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:03.027142 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.027482 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.027564 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:46:03.027922 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.018441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:40845 (local address 127.19.74.193:41903)
0806 22:46:03.018722 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.018761 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.019231 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.019440 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.019444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.020015 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:03.020213 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.020217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.020885 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.020886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.021678 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.021679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.022022 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.022023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.022137 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.022568 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:03.022577 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:03.024348 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:03.026272 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:03.026279 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:03.026281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:03.026497 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:03.026740 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:03.026741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:03.026742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:03.026900 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:03.027150 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:03.027156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:03.027331 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.027446 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.027632 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":117,"threads_started":1}
I20260806 22:46:03.028458 6098 heartbeater.cc:347] Connected to a master server at 127.19.74.254:40845
I20260806 22:46:03.028615 6098 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:03.028812 6098 heartbeater.cc:511] Master 127.19.74.254:40845 requested a full tablet report, sending...
I20260806 22:46:03.029337 5900 ts_manager.cc:194] Registered new tserver with Master: 0c9ca5f270e7489296e34ffd140254e4 (127.19.74.193:39121)
I20260806 22:46:03.029934 5900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:41903
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:46:03.051584 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:40845
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:03.143194 6103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:03.143405 6103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:03.143430 6103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:03.144600 6103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:03.144640 6103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:03.144665 6103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:46:03.146032 6103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:40845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:03.146237 6103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:03.146414 6103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:03.151885 6109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:03.151949 6112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:03.152066 6103 server_base.cc:1034] running on GCE node
W20260806 22:46:03.151916 6110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:03.152529 6103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:03.152756 6103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:03.153908 6103 hybrid_clock.cc:648] HybridClock initialized: now 1786056363153888 us; error 32 us; skew 500 ppm
Aug 06 22:46:03 dist-test-slave-pgkr krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056363, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:03.156617 6103 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:46:03.157071 6103 webserver.cc:460] Webserver started at http://127.19.74.194:35335/ using document root <none> and password file <none>
I20260806 22:46:03.157317 6103 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:03.157361 6103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:03.157516 6103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:03.158594 6103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "280077aeae34482e9519f14dbe203ff0"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "275e302302dde416a32d360fd094087c"
server_key_iv: "602c6e3402fac05ce670d7faa57e982b"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.159035 6103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "280077aeae34482e9519f14dbe203ff0"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "275e302302dde416a32d360fd094087c"
server_key_iv: "602c6e3402fac05ce670d7faa57e982b"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.160429 6103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:03.161268 6119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.161455 6103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:03.161554 6103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "280077aeae34482e9519f14dbe203ff0"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "275e302302dde416a32d360fd094087c"
server_key_iv: "602c6e3402fac05ce670d7faa57e982b"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.161629 6103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:03.182327 6103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:03.184401 6103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:03.184578 6103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:03.184828 6103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:03.185173 6103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:03.185209 6103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.185230 6103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:03.185279 6103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.191496 6103 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:38223
I20260806 22:46:03.191547 6236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:38223 every 8 connection(s)
I20260806 22:46:03.191866 6103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:46:03 dist-test-slave-pgkr krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056363, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:03.197944 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 6103
I20260806 22:46:03.198058 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:03.198313 19755 external_mini_cluster.cc:1428] Setting key 0d741a0928f7ce3c89071c25fabe2256
I20260806 22:46:03.201411 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.192585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:44151 (local address 127.19.74.254:40845)
0806 22:46:03.192653 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.192656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.192889 (+ 233us) server_negotiation.cc:407] Connection header received
0806 22:46:03.193380 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.193381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.193390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.193438 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:03.194355 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.194925 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.195892 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.196075 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.196138 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.198298 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:03.198300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:03.198300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:03.198308 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:03.199820 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:03.200323 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:03.200324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:03.200324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:03.200376 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:03.200662 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:03.200663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:03.200663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:03.200760 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:03.200811 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.201167 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.201260 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:46:03.201553 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.192574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:40845 (local address 127.19.74.194:44151)
0806 22:46:03.192763 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.192801 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.193295 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.193499 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.193502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.194083 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:03.194283 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.194287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.195002 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.195003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.195820 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.195822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.196155 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.196155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.196255 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.196700 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:03.196708 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:03.198208 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:03.199903 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:03.199910 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:03.199913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:03.200246 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:03.200440 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:03.200441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:03.200441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:03.200588 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:03.200825 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:03.200830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:03.201000 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.201129 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.201298 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260806 22:46:03.201967 6237 heartbeater.cc:347] Connected to a master server at 127.19.74.254:40845
I20260806 22:46:03.202082 6237 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:03.202271 6237 heartbeater.cc:511] Master 127.19.74.254:40845 requested a full tablet report, sending...
I20260806 22:46:03.202697 5900 ts_manager.cc:194] Registered new tserver with Master: 280077aeae34482e9519f14dbe203ff0 (127.19.74.194:38223)
I20260806 22:46:03.203228 5900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:44151
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:46:03.225270 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:40845
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:46673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:03.309494 6243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:03.309741 6243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:03.309774 6243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:03.311089 6243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:03.311139 6243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:03.311183 6243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:46:03.312652 6243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:46673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:40845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:03.312922 6243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:03.313170 6243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:46:03.318280 6243 server_base.cc:1034] running on GCE node
W20260806 22:46:03.318301 6248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:03.318301 6251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:03.318392 6249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:03.318719 6243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:03.318955 6243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:03.320122 6243 hybrid_clock.cc:648] HybridClock initialized: now 1786056363320110 us; error 25 us; skew 500 ppm
Aug 06 22:46:03 dist-test-slave-pgkr krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056363, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:03.322222 6243 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:46:03.322685 6243 webserver.cc:460] Webserver started at http://127.19.74.195:46337/ using document root <none> and password file <none>
I20260806 22:46:03.322924 6243 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:03.322993 6243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:03.323143 6243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:03.324110 6243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "45741e0b1f374e22a1355f8012066ae5"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "c9385cbd861db3fc0a74f19e67fb1a49"
server_key_iv: "d4c400589b230bcd3039a8cf327eb2dc"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.324481 6243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "45741e0b1f374e22a1355f8012066ae5"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "c9385cbd861db3fc0a74f19e67fb1a49"
server_key_iv: "d4c400589b230bcd3039a8cf327eb2dc"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.325762 6243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:03.326511 6259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.326723 6243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:03.326824 6243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "45741e0b1f374e22a1355f8012066ae5"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "c9385cbd861db3fc0a74f19e67fb1a49"
server_key_iv: "d4c400589b230bcd3039a8cf327eb2dc"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.326930 6243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:03.339562 6243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:03.341307 6243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:03.341464 6243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:03.341668 6243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:03.341964 6243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:03.342005 6243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.342024 6243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:03.342067 6243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.347818 6243 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:34071
I20260806 22:46:03.347864 6379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:34071 every 8 connection(s)
I20260806 22:46:03.348187 6243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:03.350490 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 6243
I20260806 22:46:03.350631 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:03.350827 19755 external_mini_cluster.cc:1428] Setting key e3127697ac3799d6205edbb44dd13063
Aug 06 22:46:03 dist-test-slave-pgkr krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056363, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:03.357044 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.348871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:51725 (local address 127.19.74.254:40845)
0806 22:46:03.348930 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.348932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.349149 (+ 217us) server_negotiation.cc:407] Connection header received
0806 22:46:03.349700 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.349701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.349715 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.349782 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:03.350828 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.351548 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.352535 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.352740 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.352826 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.354793 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:03.354794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:03.354794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:03.354800 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:03.355927 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:03.356301 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:03.356302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:03.356302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:03.356353 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:03.356556 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:03.356557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:03.356557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:03.356655 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:03.356705 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.356924 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.356982 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:46:03.357213 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.348871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:40845 (local address 127.19.74.195:51725)
0806 22:46:03.349029 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.349062 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.349571 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.349857 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.349861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.350442 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:03.350713 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.350719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.351634 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.351635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.352444 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.352446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.352835 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.352837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.352970 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.353367 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:03.353375 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:03.354717 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:03.355995 (+ 1278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:03.356001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:03.356003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:03.356235 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:03.356410 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:03.356410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:03.356411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:03.356504 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:03.356707 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:03.356711 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:03.356818 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.356898 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.357011 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260806 22:46:03.357692 6380 heartbeater.cc:347] Connected to a master server at 127.19.74.254:40845
I20260806 22:46:03.357779 6380 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:03.357911 6380 heartbeater.cc:511] Master 127.19.74.254:40845 requested a full tablet report, sending...
I20260806 22:46:03.358227 5900 ts_manager.cc:194] Registered new tserver with Master: 45741e0b1f374e22a1355f8012066ae5 (127.19.74.195:34071)
I20260806 22:46:03.358544 5900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:51725
I20260806 22:46:03.362504 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:46:03.369406 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.363812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.19.74.254:40845)
0806 22:46:03.363913 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.363916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.363988 (+ 72us) server_negotiation.cc:407] Connection header received
0806 22:46:03.364025 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.364025 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.364033 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.364085 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:03.364670 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.365138 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.365843 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.366008 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.366059 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.367314 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:03.367315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:03.367315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:03.367321 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:03.368419 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:03.368769 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:03.368769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:03.368769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:03.368820 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:03.368968 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:03.368968 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:03.368969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:03.369075 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:03.369161 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.369297 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.369347 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:46:03.370851 5900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:03.372386 5900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:03.378684 6393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.374643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:34071 (local address 127.0.0.1:37442)
0806 22:46:03.374817 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.374854 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.374920 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.375204 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.375207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.375209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:03.375284 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.375286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.376356 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.376357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.377489 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.377492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.378346 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.378348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.378429 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.378441 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.378535 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.378591 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":67,"threads_started":1}
I20260806 22:46:03.378703 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.374819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.19.74.195:34071)
0806 22:46:03.375002 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.375007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.375016 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:03.375054 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.375056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.375074 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.375364 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:03.375427 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.376243 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.377584 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.378259 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.378344 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.378402 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.378553 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.378591 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
I20260806 22:46:03.378858 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.374952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:39121 (local address 127.0.0.1:35782)
0806 22:46:03.375089 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.375099 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.375155 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.375486 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.375487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.375488 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:03.375523 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.375525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.376851 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.376853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.377717 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.377720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.378574 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.378575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.378656 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.378660 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.378713 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.378757 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260806 22:46:03.379122 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.374788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35782 (local address 127.19.74.193:39121)
0806 22:46:03.375136 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.375140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.375152 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:46:03.375226 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.375228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.375245 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.375348 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:03.375618 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.376722 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.377783 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.378417 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.378488 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.378537 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.378971 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.379027 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":84,"threads_started":1}
I20260806 22:46:03.379479 6310 tablet_service.cc:1467] Processing CreateTablet for tablet 02b8fae2e1c746afadd80d2de0c3d606 (DEFAULT_TABLE table=test-table [id=ca241616ef8a432685a6697c4ee4bcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:03.379762 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.374836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47014 (local address 127.19.74.194:38223)
0806 22:46:03.375033 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.375038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.375183 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:46:03.375227 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.375230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.375245 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.375330 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:03.375544 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.376726 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.378075 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.379138 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.379415 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.379474 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.379607 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.379662 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":86,"threads_started":1}
I20260806 22:46:03.379912 6310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02b8fae2e1c746afadd80d2de0c3d606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:03.379815 6031 tablet_service.cc:1467] Processing CreateTablet for tablet 02b8fae2e1c746afadd80d2de0c3d606 (DEFAULT_TABLE table=test-table [id=ca241616ef8a432685a6697c4ee4bcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:03.379781 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.374646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:38223 (local address 127.0.0.1:47014)
0806 22:46:03.375059 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.375068 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.375118 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.375339 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.375340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.375342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:03.375388 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.375390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.376848 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.376849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.377988 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.377991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.379435 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.379436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.379508 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.379512 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.379576 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.379637 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":49,"spinlock_wait_cycles":384,"thread_start_us":78,"threads_started":1}
I20260806 22:46:03.380169 6031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02b8fae2e1c746afadd80d2de0c3d606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:03.380682 6168 tablet_service.cc:1467] Processing CreateTablet for tablet 02b8fae2e1c746afadd80d2de0c3d606 (DEFAULT_TABLE table=test-table [id=ca241616ef8a432685a6697c4ee4bcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:03.380997 6168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02b8fae2e1c746afadd80d2de0c3d606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:03.382871 6398 tablet_bootstrap.cc:492] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4: Bootstrap starting.
I20260806 22:46:03.383297 6399 tablet_bootstrap.cc:492] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Bootstrap starting.
I20260806 22:46:03.383567 6398 tablet_bootstrap.cc:654] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:03.383981 6399 tablet_bootstrap.cc:654] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:03.384176 6400 tablet_bootstrap.cc:492] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Bootstrap starting.
I20260806 22:46:03.384244 6398 log.cc:824] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:03.384601 6399 log.cc:824] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:03.384828 6400 tablet_bootstrap.cc:654] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:03.385138 6398 tablet_bootstrap.cc:492] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4: No bootstrap required, opened a new log
I20260806 22:46:03.385195 6398 ts_tablet_manager.cc:1397] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:46:03.385388 6400 log.cc:824] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:03.385530 6399 tablet_bootstrap.cc:492] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: No bootstrap required, opened a new log
I20260806 22:46:03.385673 6399 ts_tablet_manager.cc:1397] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:03.386216 6400 tablet_bootstrap.cc:492] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: No bootstrap required, opened a new log
I20260806 22:46:03.386291 6400 ts_tablet_manager.cc:1397] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:03.387375 6399 raft_consensus.cc:348] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.387338 6398 raft_consensus.cc:348] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.387533 6398 raft_consensus.cc:374] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:03.387532 6399 raft_consensus.cc:374] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:03.387579 6398 raft_consensus.cc:729] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9ca5f270e7489296e34ffd140254e4, State: Initialized, Role: FOLLOWER
I20260806 22:46:03.387579 6399 raft_consensus.cc:729] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45741e0b1f374e22a1355f8012066ae5, State: Initialized, Role: FOLLOWER
I20260806 22:46:03.387696 6399 consensus_queue.cc:260] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.387727 6398 consensus_queue.cc:260] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.387995 6398 ts_tablet_manager.cc:1428] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:46:03.388000 6399 ts_tablet_manager.cc:1428] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:46:03.388134 6098 heartbeater.cc:503] Master 127.19.74.254:40845 was elected leader, sending a full tablet report...
I20260806 22:46:03.388168 6380 heartbeater.cc:503] Master 127.19.74.254:40845 was elected leader, sending a full tablet report...
I20260806 22:46:03.388311 6400 raft_consensus.cc:348] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.388509 6400 raft_consensus.cc:374] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:03.388559 6400 raft_consensus.cc:729] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280077aeae34482e9519f14dbe203ff0, State: Initialized, Role: FOLLOWER
I20260806 22:46:03.388666 6400 consensus_queue.cc:260] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.388945 6237 heartbeater.cc:503] Master 127.19.74.254:40845 was elected leader, sending a full tablet report...
I20260806 22:46:03.389362 6400 ts_tablet_manager.cc:1428] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 22:46:03.442534 6238 tablet.cc:2369] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:46:03.518534 6099 tablet.cc:2369] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:03.587497 6405 raft_consensus.cc:482] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:03.587643 6405 raft_consensus.cc:504] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.587963 6405 leader_election.cc:288] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45741e0b1f374e22a1355f8012066ae5 (127.19.74.195:34071), 280077aeae34482e9519f14dbe203ff0 (127.19.74.194:38223)
I20260806 22:46:03.592064 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.588270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:59355 (local address 127.19.74.195:34071)
0806 22:46:03.588314 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.588316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.588379 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:46:03.588445 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.588446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.588457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.588534 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:03.588779 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.589523 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.591112 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.591700 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.591774 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.591782 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.591955 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.591993 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:46:03.592064 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.588262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:51395 (local address 127.19.74.194:38223)
0806 22:46:03.588314 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.588316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.588367 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:46:03.588447 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.588448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.588460 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.588534 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:03.588780 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.589528 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.591118 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.591691 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.591774 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.591782 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.591955 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.591993 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:46:03.592106 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.588167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:34071 (local address 127.19.74.193:59355)
0806 22:46:03.588300 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.588312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.588380 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.588552 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.588553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.588557 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:03.588706 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.588707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.589612 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.589612 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.591019 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.591023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.591786 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.591787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.591868 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.591874 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.591934 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.591981 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":73,"thread_start_us":64,"threads_started":1}
I20260806 22:46:03.592106 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.588169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:38223 (local address 127.19.74.193:51395)
0806 22:46:03.588300 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.588312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.588380 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.588552 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.588553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.588556 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:03.588687 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.588693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.589611 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.589612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.591019 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.591023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.591783 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.591785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.591868 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.591875 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.591934 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.591981 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":139,"thread_start_us":62,"threads_started":1}
I20260806 22:46:03.592594 6189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02b8fae2e1c746afadd80d2de0c3d606" candidate_uuid: "0c9ca5f270e7489296e34ffd140254e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280077aeae34482e9519f14dbe203ff0" is_pre_election: true
I20260806 22:46:03.592593 6330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02b8fae2e1c746afadd80d2de0c3d606" candidate_uuid: "0c9ca5f270e7489296e34ffd140254e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45741e0b1f374e22a1355f8012066ae5" is_pre_election: true
I20260806 22:46:03.592762 6330 raft_consensus.cc:2454] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c9ca5f270e7489296e34ffd140254e4 in term 0.
I20260806 22:46:03.592762 6189 raft_consensus.cc:2454] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c9ca5f270e7489296e34ffd140254e4 in term 0.
I20260806 22:46:03.592996 5984 leader_election.cc:302] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c9ca5f270e7489296e34ffd140254e4, 280077aeae34482e9519f14dbe203ff0; no voters:
I20260806 22:46:03.593146 6405 raft_consensus.cc:2781] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:03.593225 6405 raft_consensus.cc:482] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:03.593242 6405 raft_consensus.cc:3037] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:03.593995 6405 raft_consensus.cc:504] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.594142 6405 leader_election.cc:288] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [CANDIDATE]: Term 1 election: Requested vote from peers 45741e0b1f374e22a1355f8012066ae5 (127.19.74.195:34071), 280077aeae34482e9519f14dbe203ff0 (127.19.74.194:38223)
I20260806 22:46:03.594313 6189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02b8fae2e1c746afadd80d2de0c3d606" candidate_uuid: "0c9ca5f270e7489296e34ffd140254e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280077aeae34482e9519f14dbe203ff0"
I20260806 22:46:03.594313 6330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02b8fae2e1c746afadd80d2de0c3d606" candidate_uuid: "0c9ca5f270e7489296e34ffd140254e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45741e0b1f374e22a1355f8012066ae5"
I20260806 22:46:03.594412 6189 raft_consensus.cc:3037] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:03.594412 6330 raft_consensus.cc:3037] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:03.595371 6189 raft_consensus.cc:2454] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c9ca5f270e7489296e34ffd140254e4 in term 1.
I20260806 22:46:03.595371 6330 raft_consensus.cc:2454] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c9ca5f270e7489296e34ffd140254e4 in term 1.
I20260806 22:46:03.595558 5981 leader_election.cc:302] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c9ca5f270e7489296e34ffd140254e4, 45741e0b1f374e22a1355f8012066ae5; no voters:
I20260806 22:46:03.595665 6405 raft_consensus.cc:2781] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:03.595881 6405 raft_consensus.cc:686] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 1 LEADER]: Becoming Leader. State: Replica: 0c9ca5f270e7489296e34ffd140254e4, State: Running, Role: LEADER
I20260806 22:46:03.595986 6405 consensus_queue.cc:237] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } }
I20260806 22:46:03.596909 5899 catalog_manager.cc:5495] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c9ca5f270e7489296e34ffd140254e4 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "0c9ca5f270e7489296e34ffd140254e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c9ca5f270e7489296e34ffd140254e4" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 39121 } health_report { overall_health: HEALTHY } } }
W20260806 22:46:03.598729 6381 tablet.cc:2369] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:03.607355 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.605259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35784 (local address 127.19.74.193:39121)
0806 22:46:03.605299 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.605301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.605319 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:46:03.605381 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.605381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.605397 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.605454 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:03.605560 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.605988 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.606709 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.606883 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.606935 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.607065 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:03.607153 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:03.607221 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.607268 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.607290 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:46:03.609063 6330 raft_consensus.cc:1261] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 1 FOLLOWER]: Refusing update from remote peer 0c9ca5f270e7489296e34ffd140254e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:03.609088 6189 raft_consensus.cc:1261] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 1 FOLLOWER]: Refusing update from remote peer 0c9ca5f270e7489296e34ffd140254e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:03.609470 6405 consensus_queue.cc:1035] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "280077aeae34482e9519f14dbe203ff0" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:03.609632 6409 consensus_queue.cc:1035] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45741e0b1f374e22a1355f8012066ae5" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:03.612296 6412 mvcc.cc:205] Tried to move back new op lower bound from 7315686865340129280 to 7315686865290002432. Current Snapshot: MvccSnapshot[applied={T|T < 7315686865340129280}]
I20260806 22:46:03.612543 6413 mvcc.cc:205] Tried to move back new op lower bound from 7315686865340129280 to 7315686865290002432. Current Snapshot: MvccSnapshot[applied={T|T < 7315686865340129280}]
I20260806 22:46:03.615110 5899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:03.615187 5899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:03.615975 5899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37fc77ec3e134d349cf54b9178cc9182: Sending DeleteTablet for 3 replicas of tablet 02b8fae2e1c746afadd80d2de0c3d606
I20260806 22:46:03.616335 6310 tablet_service.cc:1514] Processing DeleteTablet for tablet 02b8fae2e1c746afadd80d2de0c3d606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:03 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:37442
I20260806 22:46:03.616346 6031 tablet_service.cc:1514] Processing DeleteTablet for tablet 02b8fae2e1c746afadd80d2de0c3d606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:03 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:35782
I20260806 22:46:03.616360 6168 tablet_service.cc:1514] Processing DeleteTablet for tablet 02b8fae2e1c746afadd80d2de0c3d606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:03 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:47014
I20260806 22:46:03.616586 6424 tablet_replica.cc:330] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4: stopping tablet replica
I20260806 22:46:03.616688 6423 tablet_replica.cc:330] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: stopping tablet replica
I20260806 22:46:03.616726 6424 raft_consensus.cc:2229] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:03.616791 6423 raft_consensus.cc:2229] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:03.616863 6424 raft_consensus.cc:2258] T 02b8fae2e1c746afadd80d2de0c3d606 P 0c9ca5f270e7489296e34ffd140254e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:03.616890 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 5965
I20260806 22:46:03.616925 6423 raft_consensus.cc:2258] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:03.618010 6422 tablet_replica.cc:330] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: stopping tablet replica
I20260806 22:46:03.618135 6422 raft_consensus.cc:2229] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:03.618251 6422 raft_consensus.cc:2258] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:03.618415 6423 ts_tablet_manager.cc:1905] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:03.619794 6422 ts_tablet_manager.cc:1905] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:03.620008 6423 ts_tablet_manager.cc:1918] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:03.620124 6423 log.cc:1196] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/02b8fae2e1c746afadd80d2de0c3d606
I20260806 22:46:03.620445 6423 ts_tablet_manager.cc:1939] T 02b8fae2e1c746afadd80d2de0c3d606 P 45741e0b1f374e22a1355f8012066ae5: Deleting consensus metadata
I20260806 22:46:03.620837 5883 catalog_manager.cc:4841] TS 45741e0b1f374e22a1355f8012066ae5 (127.19.74.195:34071): tablet 02b8fae2e1c746afadd80d2de0c3d606 (table test-table [id=ca241616ef8a432685a6697c4ee4bcc2]) successfully deleted
I20260806 22:46:03.621838 6422 ts_tablet_manager.cc:1918] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:03.621917 6422 log.cc:1196] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/02b8fae2e1c746afadd80d2de0c3d606
I20260806 22:46:03.622205 6422 ts_tablet_manager.cc:1939] T 02b8fae2e1c746afadd80d2de0c3d606 P 280077aeae34482e9519f14dbe203ff0: Deleting consensus metadata
I20260806 22:46:03.622668 5886 catalog_manager.cc:4841] TS 280077aeae34482e9519f14dbe203ff0 (127.19.74.194:38223): tablet 02b8fae2e1c746afadd80d2de0c3d606 (table test-table [id=ca241616ef8a432685a6697c4ee4bcc2]) successfully deleted
W20260806 22:46:03.624866 5885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:39121 (error 108)
I20260806 22:46:03.625362 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.625232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:39121 (local address 127.0.0.1:35786)
0806 22:46:03.625287 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.625325 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:46:03.625396 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 6103
W20260806 22:46:03.625516 5885 catalog_manager.cc:4584] TS 0c9ca5f270e7489296e34ffd140254e4 (127.19.74.193:39121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02b8fae2e1c746afadd80d2de0c3d606: Network error: Client connection negotiation failed: client connection to 127.19.74.193:39121: connect: Connection refused (error 111)
I20260806 22:46:03.631892 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 6243
I20260806 22:46:03.638888 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 5867
2026-08-06T22:46:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2124 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T22:46:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:03Z Disabled control of system clock
I20260806 22:46:03.670786 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37653
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:37653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:03.751870 6429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:03.752084 6429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:03.752111 6429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:03.753288 6429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:03.753332 6429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:03.753345 6429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:03.753357 6429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:03.753408 6429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:03.754838 6429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:37653
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:03.755077 6429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:03.755388 6429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:03.760196 6435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:03.760342 6429 server_base.cc:1034] running on GCE node
W20260806 22:46:03.760202 6436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:03.760206 6438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:03.760811 6429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:03.761067 6429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:03.762217 6429 hybrid_clock.cc:648] HybridClock initialized: now 1786056363762208 us; error 52 us; skew 500 ppm
I20260806 22:46:03.763481 6429 webserver.cc:460] Webserver started at http://127.19.74.254:45533/ using document root <none> and password file <none>
I20260806 22:46:03.763742 6429 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:03.763790 6429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:03.763963 6429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:03.765028 6429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "de57bba285fe4f05b5f418fc6a5d31fa"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "add87a54f774dc8c2a9034d47d72c68b"
server_key_iv: "04b4d9ad6481fb9d944e753960175697"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.765403 6429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "de57bba285fe4f05b5f418fc6a5d31fa"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "add87a54f774dc8c2a9034d47d72c68b"
server_key_iv: "04b4d9ad6481fb9d944e753960175697"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.766800 6429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:03.767518 6444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.767693 6429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:03.767802 6429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "de57bba285fe4f05b5f418fc6a5d31fa"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "add87a54f774dc8c2a9034d47d72c68b"
server_key_iv: "04b4d9ad6481fb9d944e753960175697"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.767874 6429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:03.781534 6429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:03.781891 6429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:03.782045 6429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:03.787420 6429 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:37653
I20260806 22:46:03.787514 6497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:37653 every 8 connection(s)
I20260806 22:46:03.787865 6429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:03.788825 6498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:03.792186 6498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa: Bootstrap starting.
I20260806 22:46:03.792976 6498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:03.793385 6498 log.cc:824] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:03.794224 6498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa: No bootstrap required, opened a new log
I20260806 22:46:03.795384 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 6429
I20260806 22:46:03.795491 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:03.795702 19755 external_mini_cluster.cc:1428] Setting key 87f2507edd5ef6a600ba1efe5758eca1
I20260806 22:46:03.797112 6498 raft_consensus.cc:348] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57bba285fe4f05b5f418fc6a5d31fa" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37653 } }
I20260806 22:46:03.797315 6498 raft_consensus.cc:374] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:03.797364 6498 raft_consensus.cc:729] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de57bba285fe4f05b5f418fc6a5d31fa, State: Initialized, Role: FOLLOWER
I20260806 22:46:03.797504 6498 consensus_queue.cc:260] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57bba285fe4f05b5f418fc6a5d31fa" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37653 } }
I20260806 22:46:03.797600 6498 raft_consensus.cc:388] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:03.797633 6498 raft_consensus.cc:482] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:03.797679 6498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:03.798421 6498 raft_consensus.cc:504] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57bba285fe4f05b5f418fc6a5d31fa" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37653 } }
I20260806 22:46:03.798558 6498 leader_election.cc:302] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de57bba285fe4f05b5f418fc6a5d31fa; no voters:
I20260806 22:46:03.798807 6498 leader_election.cc:288] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:03.798835 6504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:03.798944 6504 raft_consensus.cc:686] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [term 1 LEADER]: Becoming Leader. State: Replica: de57bba285fe4f05b5f418fc6a5d31fa, State: Running, Role: LEADER
I20260806 22:46:03.799027 6504 consensus_queue.cc:237] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57bba285fe4f05b5f418fc6a5d31fa" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37653 } }
I20260806 22:46:03.799131 6498 sys_catalog.cc:564] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:03.799382 6506 sys_catalog.cc:455] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de57bba285fe4f05b5f418fc6a5d31fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57bba285fe4f05b5f418fc6a5d31fa" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37653 } } }
I20260806 22:46:03.799522 6506 sys_catalog.cc:458] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:03.799729 6507 sys_catalog.cc:455] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader de57bba285fe4f05b5f418fc6a5d31fa. Latest consensus state: current_term: 1 leader_uuid: "de57bba285fe4f05b5f418fc6a5d31fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57bba285fe4f05b5f418fc6a5d31fa" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37653 } } }
I20260806 22:46:03.799778 6510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:03.799880 6507 sys_catalog.cc:458] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:03.800422 6510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:03.803793 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.796247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.19.74.254:37653)
0806 22:46:03.796468 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.796477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.796535 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:46:03.797081 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.797090 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.797182 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.797357 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:03.797780 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.799671 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.800816 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.801469 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.801648 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.802423 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:03.802426 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:03.802427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:03.802674 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:03.802717 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:03.802732 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:03.803474 (+ 742us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.803506 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.803596 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260806 22:46:03.804299 6510 catalog_manager.cc:1347] Generated new cluster ID: 11b7464870474e84b42d0dd850f052cb
I20260806 22:46:03.804363 6510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:03.833227 6510 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:03.833823 6510 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:03.837532 6510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa: Generated new TSK 0
I20260806 22:46:03.837754 6510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:03.841427 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37653
--builtin_ntp_servers=127.19.74.212:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:03.949927 6525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:03.950172 6525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:03.950210 6525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:03.951442 6525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:03.951503 6525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:03.951552 6525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:46:03.952919 6525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:03.953178 6525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:03.953467 6525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:03.959363 6532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:03.959352 6534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:03.959338 6531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:03.959384 6525 server_base.cc:1034] running on GCE node
I20260806 22:46:03.959777 6525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:03.960016 6525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:03.961203 6525 hybrid_clock.cc:648] HybridClock initialized: now 1786056363961190 us; error 34 us; skew 500 ppm
I20260806 22:46:03.966455 6525 webserver.cc:460] Webserver started at http://127.19.74.193:44919/ using document root <none> and password file <none>
I20260806 22:46:03.966764 6525 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:03.966837 6525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:03.966995 6525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:03.968081 6525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "d84c708d7b85cfe007e7036e35c3a5f4"
server_key_iv: "3680a0426633469ff4c576633e391b71"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.968452 6525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "d84c708d7b85cfe007e7036e35c3a5f4"
server_key_iv: "3680a0426633469ff4c576633e391b71"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.969797 6525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:03.970542 6540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.970748 6525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:03.970840 6525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733"
format_stamp: "Formatted at 2026-08-06 22:46:03 on dist-test-slave-pgkr"
server_key: "d84c708d7b85cfe007e7036e35c3a5f4"
server_key_iv: "3680a0426633469ff4c576633e391b71"
server_key_version: "encryptionkey@0"
I20260806 22:46:03.970894 6525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:03.982187 6525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:03.982422 6525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:03.982517 6525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:03.982790 6525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:03.983073 6525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:03.983103 6525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.983124 6525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:03.983135 6525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:03.988967 6525 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:41895
I20260806 22:46:03.989027 6657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:41895 every 8 connection(s)
I20260806 22:46:03.989356 6525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:03.994185 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.990064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:56955 (local address 127.19.74.254:37653)
0806 22:46:03.990163 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:03.990165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:03.990363 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:46:03.990895 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:03.990896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:03.990909 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:03.990961 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:03.991565 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.992064 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.992984 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.993175 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.993225 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.993938 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:03.993939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:03.993939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:03.993944 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:03.993958 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:03.993963 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:03.994004 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:03.994104 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:46:03.994130 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:46:03.994345 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:03.990028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37653 (local address 127.19.74.193:56955)
0806 22:46:03.990242 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:03.990275 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:03.990810 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:03.991011 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:03.991014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:03.991022 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:03.991497 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.991501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.992144 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.992144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.992908 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:03.992910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:03.993248 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:03.993249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:03.993359 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:03.993808 (+ 449us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:03.993808 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:03.993835 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:03.993856 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:03.994016 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:03.994020 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:03.994022 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:46:03.994084 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:46:03.994159 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260806 22:46:03.994890 6658 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37653
I20260806 22:46:03.994967 6658 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:03.995110 6658 heartbeater.cc:511] Master 127.19.74.254:37653 requested a full tablet report, sending...
I20260806 22:46:03.995568 6461 ts_manager.cc:194] Registered new tserver with Master: 70e61e7e7f2c4f59b7ee6f2e37ef4733 (127.19.74.193:41895)
I20260806 22:46:03.996202 6461 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.193:56955
I20260806 22:46:03.998376 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 6525
I20260806 22:46:03.998509 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:03.998699 19755 external_mini_cluster.cc:1428] Setting key f2665aa751afe5ca2dcd29441fe98fde
I20260806 22:46:03.999640 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37653
--builtin_ntp_servers=127.19.74.212:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:04.081806 6661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:04.082010 6661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:04.082032 6661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:04.083254 6661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:04.083295 6661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:04.083320 6661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:46:04.084676 6661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:04.084915 6661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:04.085232 6661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:04.090196 6666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:04.090214 6669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:04.090215 6667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:04.090569 6661 server_base.cc:1034] running on GCE node
I20260806 22:46:04.090816 6661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:04.091013 6661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:04.092190 6661 hybrid_clock.cc:648] HybridClock initialized: now 1786056364092173 us; error 26 us; skew 500 ppm
I20260806 22:46:04.093423 6661 webserver.cc:460] Webserver started at http://127.19.74.194:35431/ using document root <none> and password file <none>
I20260806 22:46:04.093683 6661 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:04.093753 6661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:04.093906 6661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:04.095016 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "82f20e66980a4d11b020710418734a13"
format_stamp: "Formatted at 2026-08-06 22:46:04 on dist-test-slave-pgkr"
server_key: "c819176ae020267b1b52053b64b7e677"
server_key_iv: "e019d483b59accb166f0e609aad365d2"
server_key_version: "encryptionkey@0"
I20260806 22:46:04.095374 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "82f20e66980a4d11b020710418734a13"
format_stamp: "Formatted at 2026-08-06 22:46:04 on dist-test-slave-pgkr"
server_key: "c819176ae020267b1b52053b64b7e677"
server_key_iv: "e019d483b59accb166f0e609aad365d2"
server_key_version: "encryptionkey@0"
I20260806 22:46:04.096674 6661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:46:04.097357 6676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:04.097535 6661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:04.097647 6661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "82f20e66980a4d11b020710418734a13"
format_stamp: "Formatted at 2026-08-06 22:46:04 on dist-test-slave-pgkr"
server_key: "c819176ae020267b1b52053b64b7e677"
server_key_iv: "e019d483b59accb166f0e609aad365d2"
server_key_version: "encryptionkey@0"
I20260806 22:46:04.097718 6661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:04.131858 6661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:04.132210 6661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:04.132397 6661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:04.132658 6661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:04.133019 6661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:04.133076 6661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:04.133123 6661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:04.133148 6661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:04.139204 6661 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:32923
I20260806 22:46:04.139266 6796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:32923 every 8 connection(s)
I20260806 22:46:04.139647 6661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:04.144706 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.140460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:44793 (local address 127.19.74.254:37653)
0806 22:46:04.140530 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.140532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.140766 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:46:04.141247 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.141248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.141258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.141325 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:04.141973 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.142465 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.143492 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.143680 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.143727 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.144470 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:04.144472 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:04.144472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:04.144476 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:04.144491 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:04.144496 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:04.144539 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.144621 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.144646 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:46:04.144942 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.140438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37653 (local address 127.19.74.194:44793)
0806 22:46:04.140631 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:04.140674 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:04.141162 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:04.141382 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:04.141385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:04.141393 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:04.141905 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.141909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.142549 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.142550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.143406 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.143409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.143754 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.143754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.143866 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.144343 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:04.144343 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:04.144370 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:04.144392 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:04.144551 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:04.144555 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:04.144557 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:46:04.144614 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:46:04.144691 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260806 22:46:04.145419 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 6661
I20260806 22:46:04.145517 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:04.145546 6797 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37653
I20260806 22:46:04.145654 6797 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:04.145767 19755 external_mini_cluster.cc:1428] Setting key e2333d40ca0a0c5131782f114e9dcc5d
I20260806 22:46:04.145864 6797 heartbeater.cc:511] Master 127.19.74.254:37653 requested a full tablet report, sending...
I20260806 22:46:04.146281 6460 ts_manager.cc:194] Registered new tserver with Master: 82f20e66980a4d11b020710418734a13 (127.19.74.194:32923)
I20260806 22:46:04.146736 6460 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.194:44793
I20260806 22:46:04.146859 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37653
--builtin_ntp_servers=127.19.74.212:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:04.229202 6800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:04.229427 6800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:04.229461 6800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:04.230535 6800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:04.230598 6800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:04.230688 6800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:46:04.232081 6800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:04.232380 6800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:04.232681 6800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:04.237799 6805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:04.237818 6800 server_base.cc:1034] running on GCE node
W20260806 22:46:04.237787 6806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:04.237773 6808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:04.238204 6800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:04.238445 6800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:04.239612 6800 hybrid_clock.cc:648] HybridClock initialized: now 1786056364239578 us; error 45 us; skew 500 ppm
I20260806 22:46:04.240869 6800 webserver.cc:460] Webserver started at http://127.19.74.195:43661/ using document root <none> and password file <none>
I20260806 22:46:04.241112 6800 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:04.241182 6800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:04.241334 6800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:04.242353 6800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "8cf56acbf7ef4a91bb6578824d0a88cb"
format_stamp: "Formatted at 2026-08-06 22:46:04 on dist-test-slave-pgkr"
server_key: "ddf71f17ceddeddb7dfcd597864bae09"
server_key_iv: "b8909a2a0808bb1224bd083730d1222b"
server_key_version: "encryptionkey@0"
I20260806 22:46:04.242746 6800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "8cf56acbf7ef4a91bb6578824d0a88cb"
format_stamp: "Formatted at 2026-08-06 22:46:04 on dist-test-slave-pgkr"
server_key: "ddf71f17ceddeddb7dfcd597864bae09"
server_key_iv: "b8909a2a0808bb1224bd083730d1222b"
server_key_version: "encryptionkey@0"
I20260806 22:46:04.244012 6800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:04.244752 6815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:04.244911 6800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:04.245009 6800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "8cf56acbf7ef4a91bb6578824d0a88cb"
format_stamp: "Formatted at 2026-08-06 22:46:04 on dist-test-slave-pgkr"
server_key: "ddf71f17ceddeddb7dfcd597864bae09"
server_key_iv: "b8909a2a0808bb1224bd083730d1222b"
server_key_version: "encryptionkey@0"
I20260806 22:46:04.245090 6800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:04.254395 6800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:04.254726 6800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:04.254891 6800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:04.255112 6800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:04.255440 6800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:04.255503 6800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:04.255545 6800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:04.255571 6800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:04.261198 6800 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:40771
I20260806 22:46:04.261237 6936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:40771 every 8 connection(s)
I20260806 22:46:04.261588 6800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:04.262526 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 6800
I20260806 22:46:04.262660 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:04.262912 19755 external_mini_cluster.cc:1428] Setting key f7dd353de4f7c7f157d6ffbdac618423
I20260806 22:46:04.266983 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.262384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:46151 (local address 127.19.74.254:37653)
0806 22:46:04.262460 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.262462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.262763 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:46:04.263470 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.263471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.263484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.263540 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:04.264297 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.264787 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.265717 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.265895 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.265946 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.266775 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:04.266776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:04.266776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:04.266780 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:04.266795 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:04.266799 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:04.266849 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.266901 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.266925 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:46:04.267217 6939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.262344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37653 (local address 127.19.74.195:46151)
0806 22:46:04.262592 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:04.262677 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:04.263355 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:04.263607 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:04.263628 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:04.263637 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:04.264211 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.264216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.264876 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.264877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.265629 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.265631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.265966 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.265967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.266070 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.266587 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:04.266588 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:04.266658 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:04.266685 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:04.266852 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:04.266854 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:04.266857 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:04.266901 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:46:04.266976 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":117,"threads_started":1}
I20260806 22:46:04.267679 6937 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37653
I20260806 22:46:04.267755 6937 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:04.267944 6937 heartbeater.cc:511] Master 127.19.74.254:37653 requested a full tablet report, sending...
I20260806 22:46:04.268316 6460 ts_manager.cc:194] Registered new tserver with Master: 8cf56acbf7ef4a91bb6578824d0a88cb (127.19.74.195:40771)
I20260806 22:46:04.268641 6460 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.195:46151
I20260806 22:46:04.274552 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:46:04.279255 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.276017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.19.74.254:37653)
0806 22:46:04.276092 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.276094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.276197 (+ 103us) server_negotiation.cc:407] Connection header received
0806 22:46:04.276298 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.276299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.276313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.276371 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:04.276571 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.277046 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.277827 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.278030 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.278133 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.278939 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:04.278941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:04.278941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:04.278946 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:04.278961 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:04.278966 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:04.279083 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.279160 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.279171 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:46:04.280944 6460 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:04.282500 6460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:04.289304 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.284942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60962 (local address 127.19.74.193:41895)
0806 22:46:04.285086 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.285090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.285321 (+ 231us) server_negotiation.cc:407] Connection header received
0806 22:46:04.285437 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.285440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.285482 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.285561 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:04.285853 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.287101 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.288397 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.288899 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.288984 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.289084 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.289178 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.289218 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260806 22:46:04.289291 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.284751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:41895 (local address 127.0.0.1:60962)
0806 22:46:04.285203 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:04.285232 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:04.285329 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:04.285654 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:04.285657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:04.285659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:04.285734 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.285736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.287207 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.287208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.288307 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.288309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.288996 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.288997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.289073 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.289087 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:46:04.289162 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:46:04.289216 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":73,"threads_started":1}
I20260806 22:46:04.289569 6951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.285184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37162 (local address 127.19.74.194:32923)
0806 22:46:04.285344 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.285348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.285734 (+ 386us) server_negotiation.cc:407] Connection header received
0806 22:46:04.285811 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.285813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.285831 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.285916 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:04.285986 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.287122 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.288297 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.288991 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.289056 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.289144 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.289418 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.289457 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260806 22:46:04.289671 6947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.284843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.19.74.195:40771)
0806 22:46:04.285047 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.285052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.285516 (+ 464us) server_negotiation.cc:407] Connection header received
0806 22:46:04.285595 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.285598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.285629 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.285805 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:04.285995 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.286785 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.288221 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.288956 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.289055 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.289143 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.289533 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.289578 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260806 22:46:04.289762 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.284963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:32923 (local address 127.0.0.1:37162)
0806 22:46:04.285611 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:04.285621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:04.285667 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:04.285907 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:04.285908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:04.285909 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:04.285939 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.285940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.287230 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.287231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.288168 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.288190 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.289187 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.289188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.289258 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.289265 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:46:04.289322 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:46:04.289369 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260806 22:46:04.289459 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.284758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:40771 (local address 127.0.0.1:37894)
0806 22:46:04.285439 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:04.285449 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:04.285516 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:04.285866 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:04.285867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:04.285869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:04.285914 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.285915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.286873 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.286874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.288127 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.288130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.289188 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.289189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.289258 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.289265 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:46:04.289322 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:46:04.289369 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":7,"spinlock_wait_cycles":768,"thread_start_us":81,"threads_started":1}
I20260806 22:46:04.290546 6727 tablet_service.cc:1467] Processing CreateTablet for tablet 119010d0631e4c66ad97dbea79be9f61 (DEFAULT_TABLE table=test-table [id=342f414b526d4d31937e9409ca75ba7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:04.290548 6868 tablet_service.cc:1467] Processing CreateTablet for tablet 119010d0631e4c66ad97dbea79be9f61 (DEFAULT_TABLE table=test-table [id=342f414b526d4d31937e9409ca75ba7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:04.290539 6590 tablet_service.cc:1467] Processing CreateTablet for tablet 119010d0631e4c66ad97dbea79be9f61 (DEFAULT_TABLE table=test-table [id=342f414b526d4d31937e9409ca75ba7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:04.290957 6868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119010d0631e4c66ad97dbea79be9f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:04.290957 6727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119010d0631e4c66ad97dbea79be9f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:04.290968 6590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119010d0631e4c66ad97dbea79be9f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:04.294414 6953 tablet_bootstrap.cc:492] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: Bootstrap starting.
I20260806 22:46:04.294746 6954 tablet_bootstrap.cc:492] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733: Bootstrap starting.
I20260806 22:46:04.294775 6952 tablet_bootstrap.cc:492] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: Bootstrap starting.
I20260806 22:46:04.295364 6954 tablet_bootstrap.cc:654] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:04.295377 6952 tablet_bootstrap.cc:654] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:04.295656 6954 log.cc:824] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:04.295691 6952 log.cc:824] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:04.296435 6953 tablet_bootstrap.cc:654] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:04.296453 6954 tablet_bootstrap.cc:492] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733: No bootstrap required, opened a new log
I20260806 22:46:04.296463 6952 tablet_bootstrap.cc:492] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: No bootstrap required, opened a new log
I20260806 22:46:04.296533 6954 ts_tablet_manager.cc:1397] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:04.296533 6952 ts_tablet_manager.cc:1397] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:04.296844 6953 log.cc:824] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:04.297626 6953 tablet_bootstrap.cc:492] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: No bootstrap required, opened a new log
I20260806 22:46:04.297719 6953 ts_tablet_manager.cc:1397] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:46:04.299233 6952 raft_consensus.cc:348] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.299226 6954 raft_consensus.cc:348] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.299399 6954 raft_consensus.cc:374] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:04.299399 6952 raft_consensus.cc:374] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:04.299437 6952 raft_consensus.cc:729] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f20e66980a4d11b020710418734a13, State: Initialized, Role: FOLLOWER
I20260806 22:46:04.299438 6954 raft_consensus.cc:729] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e61e7e7f2c4f59b7ee6f2e37ef4733, State: Initialized, Role: FOLLOWER
I20260806 22:46:04.299582 6954 consensus_queue.cc:260] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.299580 6952 consensus_queue.cc:260] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.299877 6952 ts_tablet_manager.cc:1428] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:46:04.299961 6954 ts_tablet_manager.cc:1428] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:46:04.300021 6797 heartbeater.cc:503] Master 127.19.74.254:37653 was elected leader, sending a full tablet report...
I20260806 22:46:04.300082 6658 heartbeater.cc:503] Master 127.19.74.254:37653 was elected leader, sending a full tablet report...
I20260806 22:46:04.300248 6953 raft_consensus.cc:348] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.300446 6953 raft_consensus.cc:374] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:04.300652 6953 raft_consensus.cc:729] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf56acbf7ef4a91bb6578824d0a88cb, State: Initialized, Role: FOLLOWER
I20260806 22:46:04.300803 6953 consensus_queue.cc:260] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.301084 6953 ts_tablet_manager.cc:1428] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:04.301156 6937 heartbeater.cc:503] Master 127.19.74.254:37653 was elected leader, sending a full tablet report...
I20260806 22:46:04.364094 6958 raft_consensus.cc:482] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:04.364266 6958 raft_consensus.cc:504] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.364616 6958 leader_election.cc:288] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70e61e7e7f2c4f59b7ee6f2e37ef4733 (127.19.74.193:41895), 8cf56acbf7ef4a91bb6578824d0a88cb (127.19.74.195:40771)
I20260806 22:46:04.367616 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.364790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:40771 (local address 127.19.74.194:55739)
0806 22:46:04.364966 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:04.364975 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:04.365036 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:04.365174 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:04.365175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:04.365176 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:04.365265 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.365267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.365928 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.365929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.366762 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.366765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.367345 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.367346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.367417 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.367421 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:04.367476 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:46:04.367505 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":15,"thread_start_us":93,"threads_started":1}
I20260806 22:46:04.367624 6947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.364853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:55739 (local address 127.19.74.195:40771)
0806 22:46:04.364910 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.364912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.365043 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:46:04.365090 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.365091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.365101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.365164 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:04.365345 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.365842 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.366850 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.367252 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.367317 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.367338 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.367504 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.367552 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:46:04.367691 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.364867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:32871 (local address 127.19.74.193:41895)
0806 22:46:04.364914 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.364916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.364933 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:46:04.365034 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.365035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.365048 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.365096 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:04.365273 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.365750 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.366919 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.367279 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.367375 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.367382 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.367579 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.367619 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:46:04.367724 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.364778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:41895 (local address 127.19.74.194:32871)
0806 22:46:04.364877 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:04.364891 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:04.364957 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:04.365112 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:04.365113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:04.365115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:04.365193 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.365195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.365835 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.365837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.366782 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:04.366784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.367407 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.367408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:04.367498 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.367501 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:04.367563 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:46:04.367608 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":48}
I20260806 22:46:04.368238 6888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119010d0631e4c66ad97dbea79be9f61" candidate_uuid: "82f20e66980a4d11b020710418734a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" is_pre_election: true
I20260806 22:46:04.368395 6888 raft_consensus.cc:2454] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82f20e66980a4d11b020710418734a13 in term 0.
I20260806 22:46:04.368436 6611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119010d0631e4c66ad97dbea79be9f61" candidate_uuid: "82f20e66980a4d11b020710418734a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" is_pre_election: true
I20260806 22:46:04.368633 6611 raft_consensus.cc:2454] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82f20e66980a4d11b020710418734a13 in term 0.
I20260806 22:46:04.368696 6679 leader_election.cc:302] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82f20e66980a4d11b020710418734a13, 8cf56acbf7ef4a91bb6578824d0a88cb; no voters:
I20260806 22:46:04.368854 6958 raft_consensus.cc:2781] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:04.368928 6958 raft_consensus.cc:482] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:04.368986 6958 raft_consensus.cc:3037] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:04.369673 6958 raft_consensus.cc:504] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.369809 6958 leader_election.cc:288] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [CANDIDATE]: Term 1 election: Requested vote from peers 70e61e7e7f2c4f59b7ee6f2e37ef4733 (127.19.74.193:41895), 8cf56acbf7ef4a91bb6578824d0a88cb (127.19.74.195:40771)
I20260806 22:46:04.369946 6611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119010d0631e4c66ad97dbea79be9f61" candidate_uuid: "82f20e66980a4d11b020710418734a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733"
I20260806 22:46:04.369962 6888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119010d0631e4c66ad97dbea79be9f61" candidate_uuid: "82f20e66980a4d11b020710418734a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb"
I20260806 22:46:04.370029 6888 raft_consensus.cc:3037] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:04.370026 6611 raft_consensus.cc:3037] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:04.370958 6611 raft_consensus.cc:2454] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f20e66980a4d11b020710418734a13 in term 1.
I20260806 22:46:04.370958 6888 raft_consensus.cc:2454] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f20e66980a4d11b020710418734a13 in term 1.
I20260806 22:46:04.371145 6679 leader_election.cc:302] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82f20e66980a4d11b020710418734a13, 8cf56acbf7ef4a91bb6578824d0a88cb; no voters:
I20260806 22:46:04.371272 6958 raft_consensus.cc:2781] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:04.371414 6958 raft_consensus.cc:686] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 1 LEADER]: Becoming Leader. State: Replica: 82f20e66980a4d11b020710418734a13, State: Running, Role: LEADER
I20260806 22:46:04.371518 6958 consensus_queue.cc:237] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } }
I20260806 22:46:04.372361 6460 catalog_manager.cc:5495] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82f20e66980a4d11b020710418734a13 (127.19.74.194). New cstate: current_term: 1 leader_uuid: "82f20e66980a4d11b020710418734a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82f20e66980a4d11b020710418734a13" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 32923 } health_report { overall_health: HEALTHY } } }
I20260806 22:46:04.384508 6951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.382369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37178 (local address 127.19.74.194:32923)
0806 22:46:04.382422 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:04.382424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:04.382431 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:46:04.382491 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:04.382492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:04.382509 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:04.382560 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:04.382730 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.383149 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.383816 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:04.383980 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:04.384028 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:04.384157 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:04.384242 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:04.384327 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:04.384375 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 22:46:04.384384 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:46:04.386224 6611 raft_consensus.cc:1261] T 119010d0631e4c66ad97dbea79be9f61 P 70e61e7e7f2c4f59b7ee6f2e37ef4733 [term 1 FOLLOWER]: Refusing update from remote peer 82f20e66980a4d11b020710418734a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:04.386240 6888 raft_consensus.cc:1261] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 1 FOLLOWER]: Refusing update from remote peer 82f20e66980a4d11b020710418734a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:04.386566 6958 consensus_queue.cc:1035] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e61e7e7f2c4f59b7ee6f2e37ef4733" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 41895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:04.386718 6962 consensus_queue.cc:1035] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cf56acbf7ef4a91bb6578824d0a88cb" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 40771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:04.389837 6966 mvcc.cc:205] Tried to move back new op lower bound from 7315686868523311104 to 7315686868466479104. Current Snapshot: MvccSnapshot[applied={T|T < 7315686868523311104}]
I20260806 22:46:04.391192 6461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:04.391309 6461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:04.392073 6461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de57bba285fe4f05b5f418fc6a5d31fa: Sending DeleteTablet for 3 replicas of tablet 119010d0631e4c66ad97dbea79be9f61
I20260806 22:46:04.392477 6590 tablet_service.cc:1514] Processing DeleteTablet for tablet 119010d0631e4c66ad97dbea79be9f61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:04 UTC) from {username='slave'} at 127.0.0.1:60962
I20260806 22:46:04.392467 6868 tablet_service.cc:1514] Processing DeleteTablet for tablet 119010d0631e4c66ad97dbea79be9f61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:04 UTC) from {username='slave'} at 127.0.0.1:37894
I20260806 22:46:04.392686 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 6525
I20260806 22:46:04.392695 6727 tablet_service.cc:1514] Processing DeleteTablet for tablet 119010d0631e4c66ad97dbea79be9f61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:04 UTC) from {username='slave'} at 127.0.0.1:37162
I20260806 22:46:04.393188 6980 tablet_replica.cc:330] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: stopping tablet replica
I20260806 22:46:04.393499 6980 raft_consensus.cc:2229] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:04.393646 6980 raft_consensus.cc:2258] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:04.394039 6980 ts_tablet_manager.cc:1905] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:04.395782 6980 ts_tablet_manager.cc:1918] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:04.395870 6980 log.cc:1196] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/119010d0631e4c66ad97dbea79be9f61
I20260806 22:46:04.396119 6980 ts_tablet_manager.cc:1939] T 119010d0631e4c66ad97dbea79be9f61 P 82f20e66980a4d11b020710418734a13: Deleting consensus metadata
I20260806 22:46:04.396512 6447 catalog_manager.cc:4841] TS 82f20e66980a4d11b020710418734a13 (127.19.74.194:32923): tablet 119010d0631e4c66ad97dbea79be9f61 (table test-table [id=342f414b526d4d31937e9409ca75ba7f]) successfully deleted
I20260806 22:46:04.398737 6978 tablet_replica.cc:330] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: stopping tablet replica
I20260806 22:46:04.398859 6978 raft_consensus.cc:2229] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:04.398969 6978 raft_consensus.cc:2258] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:04.399369 6978 ts_tablet_manager.cc:1905] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:04.401089 6978 ts_tablet_manager.cc:1918] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:04.401185 6978 log.cc:1196] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/119010d0631e4c66ad97dbea79be9f61
I20260806 22:46:04.401475 6978 ts_tablet_manager.cc:1939] T 119010d0631e4c66ad97dbea79be9f61 P 8cf56acbf7ef4a91bb6578824d0a88cb: Deleting consensus metadata
I20260806 22:46:04.401904 6447 catalog_manager.cc:4841] TS 8cf56acbf7ef4a91bb6578824d0a88cb (127.19.74.195:40771): tablet 119010d0631e4c66ad97dbea79be9f61 (table test-table [id=342f414b526d4d31937e9409ca75ba7f]) successfully deleted
W20260806 22:46:04.403636 6446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:41895 (error 108)
I20260806 22:46:04.403903 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 6661
I20260806 22:46:04.404093 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:04.403941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:41895 (local address 127.0.0.1:60970)
0806 22:46:04.404020 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:04.404057 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:46:04.404712 6446 catalog_manager.cc:4584] TS 70e61e7e7f2c4f59b7ee6f2e37ef4733 (127.19.74.193:41895): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 119010d0631e4c66ad97dbea79be9f61: Network error: Client connection negotiation failed: client connection to 127.19.74.193:41895: connect: Connection refused (error 111)
I20260806 22:46:04.409261 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 6800
I20260806 22:46:04.415335 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 6429
2026-08-06T22:46:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (773 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:04 dist-test-slave-pgkr krb5kdc[6985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:04 dist-test-slave-pgkr krb5kdc[6985](info): set up 2 sockets
Aug 06 22:46:04 dist-test-slave-pgkr krb5kdc[6985](info): commencing operation
krb5kdc: starting...
W20260806 22:46:05.456276 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:05 dist-test-slave-pgkr krb5kdc[6985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:05.575613 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:41501
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:40773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:41501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:05.657819 6999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:05.658049 6999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:05.658087 6999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:05.659246 6999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:05.659313 6999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:05.659339 6999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:05.659372 6999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:05.659399 6999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:05.660912 6999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:40773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:41501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:41501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:05.661190 6999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:05.661414 6999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:05.666234 7005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:05.666224 7007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:05.666251 7004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:05.666478 6999 server_base.cc:1034] running on GCE node
I20260806 22:46:05.666783 6999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:05.667033 6999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:05.668180 6999 hybrid_clock.cc:648] HybridClock initialized: now 1786056365668162 us; error 33 us; skew 500 ppm
Aug 06 22:46:05 dist-test-slave-pgkr krb5kdc[6985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056365, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:05.670578 6999 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:05.671066 6999 webserver.cc:460] Webserver started at http://127.19.74.254:45485/ using document root <none> and password file <none>
I20260806 22:46:05.671267 6999 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:05.671305 6999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:05.671391 6999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:05.672402 6999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "efb1a161f808426294b0a1338486b0bb"
format_stamp: "Formatted at 2026-08-06 22:46:05 on dist-test-slave-pgkr"
server_key: "ee7faec14759044a53a73bf737f77a4c"
server_key_iv: "2228c3b525785b2aa3e46c31cca18ba9"
server_key_version: "encryptionkey@0"
I20260806 22:46:05.672713 6999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "efb1a161f808426294b0a1338486b0bb"
format_stamp: "Formatted at 2026-08-06 22:46:05 on dist-test-slave-pgkr"
server_key: "ee7faec14759044a53a73bf737f77a4c"
server_key_iv: "2228c3b525785b2aa3e46c31cca18ba9"
server_key_version: "encryptionkey@0"
I20260806 22:46:05.673961 6999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:05.674711 7014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:05.674885 6999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:05.674978 6999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "efb1a161f808426294b0a1338486b0bb"
format_stamp: "Formatted at 2026-08-06 22:46:05 on dist-test-slave-pgkr"
server_key: "ee7faec14759044a53a73bf737f77a4c"
server_key_iv: "2228c3b525785b2aa3e46c31cca18ba9"
server_key_version: "encryptionkey@0"
I20260806 22:46:05.675061 6999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:05.688335 6999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:05.690111 6999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:05.690281 6999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:05.694809 6999 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:41501
I20260806 22:46:05.694869 7067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:41501 every 8 connection(s)
I20260806 22:46:05.695228 6999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:05.696220 7068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:05.698820 7068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb: Bootstrap starting.
I20260806 22:46:05.699447 7068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:05.699759 7068 log.cc:824] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:05.700441 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 6999
I20260806 22:46:05.700536 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:05.700531 7068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb: No bootstrap required, opened a new log
I20260806 22:46:05.700730 19755 external_mini_cluster.cc:1428] Setting key c45584eb6d732e60798d11dd1ddd5066
I20260806 22:46:05.703123 7068 raft_consensus.cc:348] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb1a161f808426294b0a1338486b0bb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 41501 } }
I20260806 22:46:05.703284 7068 raft_consensus.cc:374] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:05.703349 7068 raft_consensus.cc:729] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb1a161f808426294b0a1338486b0bb, State: Initialized, Role: FOLLOWER
I20260806 22:46:05.703527 7068 consensus_queue.cc:260] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb1a161f808426294b0a1338486b0bb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 41501 } }
I20260806 22:46:05.703630 7068 raft_consensus.cc:388] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:05.703678 7068 raft_consensus.cc:482] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:05.703729 7068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:05.704279 7068 raft_consensus.cc:504] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb1a161f808426294b0a1338486b0bb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 41501 } }
I20260806 22:46:05.704413 7068 leader_election.cc:302] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efb1a161f808426294b0a1338486b0bb; no voters:
I20260806 22:46:05.704602 7068 leader_election.cc:288] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:05.704663 7073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:05.704803 7073 raft_consensus.cc:686] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [term 1 LEADER]: Becoming Leader. State: Replica: efb1a161f808426294b0a1338486b0bb, State: Running, Role: LEADER
I20260806 22:46:05.704870 7068 sys_catalog.cc:564] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:05.704973 7073 consensus_queue.cc:237] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb1a161f808426294b0a1338486b0bb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 41501 } }
I20260806 22:46:05.705395 7073 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader efb1a161f808426294b0a1338486b0bb. Latest consensus state: current_term: 1 leader_uuid: "efb1a161f808426294b0a1338486b0bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb1a161f808426294b0a1338486b0bb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 41501 } } }
I20260806 22:46:05.705403 7075 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efb1a161f808426294b0a1338486b0bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efb1a161f808426294b0a1338486b0bb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 41501 } } }
I20260806 22:46:05.705560 7073 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:05.705569 7075 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:05.705789 7083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:05.706526 7083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:05.708166 7083 catalog_manager.cc:1347] Generated new cluster ID: f2c3b7968b1c41669a811ed098f35746
I20260806 22:46:05.708214 7083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:46:05 dist-test-slave-pgkr krb5kdc[6985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:05.713168 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:05.701244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46226 (local address 127.19.74.254:41501)
0806 22:46:05.701448 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:05.701456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:05.701485 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:46:05.701932 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:05.701938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:05.701983 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:05.702092 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:05.703005 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:05.704283 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:05.705429 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:05.706106 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:05.706201 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:05.709410 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:05.709414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:05.709414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:05.709427 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:05.711452 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:05.711835 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:05.711838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:05.711839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:05.711951 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:05.712175 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:05.712176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:05.712176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:05.712513 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:05.712684 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:05.712797 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:46:05.712943 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260806 22:46:05.713824 7083 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:05.714432 7083 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:05.719811 7083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb: Generated new TSK 0
I20260806 22:46:05.719996 7083 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.74.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
Entry for principal kudu/127.19.74.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab.
I20260806 22:46:05.755458 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:41501
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:40773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:05.859572 7096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:05.859826 7096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:05.859865 7096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:05.861037 7096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:05.861109 7096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:05.861155 7096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:46:05.862809 7096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:40773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.193.keytab
--principal=kudu/127.19.74.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:05.863080 7096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:05.863312 7096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:05.868217 7104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:05.868228 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:05.868337 7101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:05.868662 7096 server_base.cc:1034] running on GCE node
I20260806 22:46:05.868891 7096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:05.869141 7096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:05.870288 7096 hybrid_clock.cc:648] HybridClock initialized: now 1786056365870275 us; error 27 us; skew 500 ppm
Aug 06 22:46:05 dist-test-slave-pgkr krb5kdc[6985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056365, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:05.873016 7096 init.cc:377] Logged in from keytab as kudu/127.19.74.193@KRBTEST.COM (short username kudu)
I20260806 22:46:05.873489 7096 webserver.cc:460] Webserver started at http://127.19.74.193:36937/ using document root <none> and password file <none>
I20260806 22:46:05.873741 7096 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:05.873809 7096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:05.873982 7096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:05.875137 7096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "86198e2492354f27a354c2dd7ea1478f"
format_stamp: "Formatted at 2026-08-06 22:46:05 on dist-test-slave-pgkr"
server_key: "378ef7344f31c52d86379075237be228"
server_key_iv: "5f89467c4240261a1640b65d123dbe69"
server_key_version: "encryptionkey@0"
I20260806 22:46:05.875504 7096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "86198e2492354f27a354c2dd7ea1478f"
format_stamp: "Formatted at 2026-08-06 22:46:05 on dist-test-slave-pgkr"
server_key: "378ef7344f31c52d86379075237be228"
server_key_iv: "5f89467c4240261a1640b65d123dbe69"
server_key_version: "encryptionkey@0"
I20260806 22:46:05.876821 7096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:05.877628 7112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:05.877812 7096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:05.877923 7096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "86198e2492354f27a354c2dd7ea1478f"
format_stamp: "Formatted at 2026-08-06 22:46:05 on dist-test-slave-pgkr"
server_key: "378ef7344f31c52d86379075237be228"
server_key_iv: "5f89467c4240261a1640b65d123dbe69"
server_key_version: "encryptionkey@0"
I20260806 22:46:05.878006 7096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:05.892716 7096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:05.894528 7096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:05.894696 7096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:05.894922 7096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:05.895231 7096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:05.895282 7096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:05.895315 7096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:05.895340 7096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:05.901609 7096 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:36099
I20260806 22:46:05.901659 7235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:36099 every 8 connection(s)
I20260806 22:46:05.901973 7096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:05.902102 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 7096
I20260806 22:46:05.902226 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:05.902712 19755 external_mini_cluster.cc:1428] Setting key 1da4dd1e651bef07ac1dba5f0951c802
Aug 06 22:46:05 dist-test-slave-pgkr krb5kdc[6985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056365, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.193@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.74.194@KRBTEST.COM; defaulting to no policy
I20260806 22:46:05.912626 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:05.902906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:55461 (local address 127.19.74.254:41501)
0806 22:46:05.902970 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:05.902972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:05.903248 (+ 276us) server_negotiation.cc:407] Connection header received
0806 22:46:05.903936 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:05.903937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:05.903947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:05.904004 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:05.905279 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:05.906137 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:05.907261 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:05.907625 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:05.907698 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:05.909710 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:05.909712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:05.909712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:05.909720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:05.911249 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:05.911649 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:05.911651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:05.911651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:05.911709 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:05.911946 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:05.911947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:05.911948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:05.912104 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:05.912172 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:05.912442 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 22:46:05.912530 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:46:05.912812 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:05.902879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:41501 (local address 127.19.74.193:55461)
0806 22:46:05.903124 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:05.903152 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:05.903831 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:05.904075 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:05.904079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:05.904903 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:05.905179 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:05.905185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:05.906226 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:05.906227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:05.907174 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:05.907177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:05.907711 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:05.907712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:05.907811 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:05.908286 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:05.908296 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:05.909621 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:05.911319 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:05.911325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:05.911327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:05.911556 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:05.911783 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:05.911783 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:05.911783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:05.911886 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:05.912167 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:05.912172 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:05.912303 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 22:46:05.912417 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:46:05.912570 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":88,"threads_started":1}
I20260806 22:46:05.913323 7236 heartbeater.cc:347] Connected to a master server at 127.19.74.254:41501
I20260806 22:46:05.913451 7236 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:05.913648 7236 heartbeater.cc:511] Master 127.19.74.254:41501 requested a full tablet report, sending...
I20260806 22:46:05.914165 7025 ts_manager.cc:194] Registered new tserver with Master: 86198e2492354f27a354c2dd7ea1478f (127.19.74.193:36099)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.194@KRBTEST.COM" created.
I20260806 22:46:05.915036 7025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.193@KRBTEST.COM'} at 127.19.74.193:55461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
Entry for principal kudu/127.19.74.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab.
I20260806 22:46:05.929369 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:41501
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:40773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:06.010470 7241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:06.010716 7241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:06.010746 7241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:06.011826 7241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:06.011861 7241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:06.011886 7241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:46:06.013263 7241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:40773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.194.keytab
--principal=kudu/127.19.74.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:06.013501 7241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:06.013684 7241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:06.018391 7247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.018482 7246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:06.018586 7241 server_base.cc:1034] running on GCE node
W20260806 22:46:06.018397 7249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:06.018944 7241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:06.019234 7241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:06.020390 7241 hybrid_clock.cc:648] HybridClock initialized: now 1786056366020367 us; error 34 us; skew 500 ppm
Aug 06 22:46:06 dist-test-slave-pgkr krb5kdc[6985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056366, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:06.022778 7241 init.cc:377] Logged in from keytab as kudu/127.19.74.194@KRBTEST.COM (short username kudu)
I20260806 22:46:06.023190 7241 webserver.cc:460] Webserver started at http://127.19.74.194:41501/ using document root <none> and password file <none>
I20260806 22:46:06.023448 7241 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:06.023514 7241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:06.023671 7241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:06.024667 7241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "c856e64d1abdfef35619313dee5c9dc9"
server_key_iv: "47b71bcbfa6f7406d523f5391ac7b565"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.025036 7241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "c856e64d1abdfef35619313dee5c9dc9"
server_key_iv: "47b71bcbfa6f7406d523f5391ac7b565"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.026381 7241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:06.027169 7256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.027345 7241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.027434 7241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "c856e64d1abdfef35619313dee5c9dc9"
server_key_iv: "47b71bcbfa6f7406d523f5391ac7b565"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.027531 7241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:06.053159 7241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:06.055305 7241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:06.055475 7241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:06.055730 7241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:06.056042 7241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:06.056093 7241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.056128 7241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:06.056154 7241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.062060 7241 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:33589
I20260806 22:46:06.062103 7377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:33589 every 8 connection(s)
I20260806 22:46:06.062422 7241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:06.064172 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 7241
I20260806 22:46:06.064292 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:06.064488 19755 external_mini_cluster.cc:1428] Setting key e27ccc673097d4d97c331b17c476b7e3
Aug 06 22:46:06 dist-test-slave-pgkr krb5kdc[6985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056366, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.194@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:06.072295 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.063135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:41501 (local address 127.19.74.194:39505)
0806 22:46:06.063275 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.063324 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.063848 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.064104 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.064108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.064882 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:06.065088 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.065092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.065760 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.065761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.066554 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.066556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.066982 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.066983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.067109 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.067625 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:06.067633 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:06.068979 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:06.070897 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:06.070903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:06.070905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:06.071127 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:06.071359 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:06.071359 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:06.071360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:06.071455 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:06.071812 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:06.071817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:06.071928 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.072009 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.072128 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260806 22:46:06.072307 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.063185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:39505 (local address 127.19.74.254:41501)
0806 22:46:06.063226 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.063229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.063406 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:46:06.063969 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.063970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.063983 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.064036 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:06.065195 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.065665 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.066672 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.066869 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.066973 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.069069 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:06.069071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:06.069071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:06.069079 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:06.070816 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:06.071212 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:06.071213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:06.071214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:06.071284 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:06.071557 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:06.071558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:06.071558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:06.071733 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:06.071808 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.072065 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.072161 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:46:06.072816 7378 heartbeater.cc:347] Connected to a master server at 127.19.74.254:41501
I20260806 22:46:06.072928 7378 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:06.073118 7378 heartbeater.cc:511] Master 127.19.74.254:41501 requested a full tablet report, sending...
I20260806 22:46:06.073539 7025 ts_manager.cc:194] Registered new tserver with Master: 145c4a8b2d38404dbfd3c9e9d4dc07a7 (127.19.74.194:33589)
I20260806 22:46:06.074072 7025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.194@KRBTEST.COM'} at 127.19.74.194:39505
WARNING: no policy specified for kudu/127.19.74.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
Entry for principal kudu/127.19.74.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab.
I20260806 22:46:06.091427 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:41501
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:40773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:06.173146 7383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:06.173334 7383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:06.173357 7383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:06.174481 7383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:06.174595 7383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:06.174669 7383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:46:06.176155 7383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:40773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.195.keytab
--principal=kudu/127.19.74.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:06.176339 7383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:06.176510 7383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:06.181399 7388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.181403 7389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.181429 7391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:06.181696 7383 server_base.cc:1034] running on GCE node
I20260806 22:46:06.181926 7383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:06.182181 7383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:06.183341 7383 hybrid_clock.cc:648] HybridClock initialized: now 1786056366183315 us; error 52 us; skew 500 ppm
Aug 06 22:46:06 dist-test-slave-pgkr krb5kdc[6985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056366, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:06.185820 7383 init.cc:377] Logged in from keytab as kudu/127.19.74.195@KRBTEST.COM (short username kudu)
I20260806 22:46:06.186301 7383 webserver.cc:460] Webserver started at http://127.19.74.195:45387/ using document root <none> and password file <none>
I20260806 22:46:06.186551 7383 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:06.186653 7383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:06.186820 7383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:06.187842 7383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "77fcf4c6ef3b4307ba9cf82799d2a005"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "3e7655a141ba88d129a6cbeb47117e21"
server_key_iv: "a201fe242eb6ccd1468fe9ec6035ff65"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.188208 7383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "77fcf4c6ef3b4307ba9cf82799d2a005"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "3e7655a141ba88d129a6cbeb47117e21"
server_key_iv: "a201fe242eb6ccd1468fe9ec6035ff65"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.189499 7383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:06.190305 7400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.190491 7383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:06.190601 7383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "77fcf4c6ef3b4307ba9cf82799d2a005"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "3e7655a141ba88d129a6cbeb47117e21"
server_key_iv: "a201fe242eb6ccd1468fe9ec6035ff65"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.190733 7383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:06.202382 7383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:06.204283 7383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:06.204434 7383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:06.204639 7383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:06.204944 7383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:06.204999 7383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.205040 7383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:06.205068 7383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.211192 7383 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:41149
I20260806 22:46:06.211225 7518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:41149 every 8 connection(s)
I20260806 22:46:06.211551 7383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:06.216471 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 7383
I20260806 22:46:06.216585 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:06.216802 19755 external_mini_cluster.cc:1428] Setting key 145c7f8b6b90a2fb038ce1c16d3b540b
Aug 06 22:46:06 dist-test-slave-pgkr krb5kdc[6985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056366, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.195@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:06.220896 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.212272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:45035 (local address 127.19.74.254:41501)
0806 22:46:06.212346 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.212349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.212539 (+ 190us) server_negotiation.cc:407] Connection header received
0806 22:46:06.212972 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.212973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.212984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.213065 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:06.214069 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.214564 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.215582 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.215771 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.215827 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.218248 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:06.218249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:06.218250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:06.218256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:06.219518 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:06.219894 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:06.219895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:06.219896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:06.219967 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:06.220234 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:06.220236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:06.220236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:06.220357 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:06.220431 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.220722 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.220807 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:46:06.221053 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.212209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:41501 (local address 127.19.74.195:45035)
0806 22:46:06.212420 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.212454 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.212891 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.213150 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.213154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.213817 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:06.213992 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.213996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.214689 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.214690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.215491 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.215494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.215847 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.215848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.215939 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.216405 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:06.216417 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:06.218155 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:06.219589 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:06.219595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:06.219597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:06.219805 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:06.220040 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:06.220041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:06.220041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:06.220155 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:06.220435 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:06.220443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:06.220575 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.220674 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.220826 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
I20260806 22:46:06.221545 7519 heartbeater.cc:347] Connected to a master server at 127.19.74.254:41501
I20260806 22:46:06.221673 7519 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:06.221859 7519 heartbeater.cc:511] Master 127.19.74.254:41501 requested a full tablet report, sending...
I20260806 22:46:06.222272 7025 ts_manager.cc:194] Registered new tserver with Master: 77fcf4c6ef3b4307ba9cf82799d2a005 (127.19.74.195:41149)
I20260806 22:46:06.222594 7025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.74.195@KRBTEST.COM'} at 127.19.74.195:45035
I20260806 22:46:06.228487 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:46:06.235294 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.229740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46238 (local address 127.19.74.254:41501)
0806 22:46:06.229809 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.229811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.229876 (+ 65us) server_negotiation.cc:407] Connection header received
0806 22:46:06.229917 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.229918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.229926 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.229971 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:06.230539 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.231010 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.231715 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.231884 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.231935 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.232951 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:06.232953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:06.232953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:06.232958 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:06.234302 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:06.234600 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:06.234601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:06.234601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:06.234706 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:06.234870 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:06.234870 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:06.234870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:06.234989 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:06.235061 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.235147 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.235202 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:46:06.236755 7025 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:06.238502 7025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:06.245076 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.240753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:33589 (local address 127.0.0.1:45338)
0806 22:46:06.240854 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.240890 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.240944 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.241284 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.241287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.241288 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:06.241351 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.241353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.242674 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.242675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.243885 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.243888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.244743 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.244743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.244847 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.244867 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.244913 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.244974 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":61,"threads_started":1}
I20260806 22:46:06.245344 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.240985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:41149 (local address 127.0.0.1:36854)
0806 22:46:06.241343 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.241353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.241405 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.241900 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.241901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.241902 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:06.241930 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.241931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.243123 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.243124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.244142 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.244144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.245078 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.245080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.245146 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.245151 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.245213 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.245255 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":74,"threads_started":1}
I20260806 22:46:06.245644 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.241041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.19.74.195:41149)
0806 22:46:06.241190 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.241193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.241761 (+ 568us) server_negotiation.cc:407] Connection header received
0806 22:46:06.241802 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.241804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.241818 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.241904 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:06.241983 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.243017 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.244223 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.244903 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.244977 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.245038 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.245509 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.245556 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260806 22:46:06.246081 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.240862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45338 (local address 127.19.74.194:33589)
0806 22:46:06.241099 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.241104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.241115 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:06.241157 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.241159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.241187 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.241266 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:06.241431 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.242540 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.243960 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.244631 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.245839 (+ 1208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.245916 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.245943 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.245993 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":98,"threads_started":1}
I20260806 22:46:06.246309 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.240760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36099 (local address 127.0.0.1:45130)
0806 22:46:06.241088 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.241098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.241155 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.241722 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.241723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.241725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:06.241776 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.241778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.242793 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.242794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.243823 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.243825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.244741 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.244742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.244847 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.246058 (+ 1211us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.246125 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.246206 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":807,"thread_start_us":70,"threads_started":1}
I20260806 22:46:06.246846 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.241158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45130 (local address 127.19.74.193:36099)
0806 22:46:06.241501 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.241505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.241516 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:06.241553 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.241554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.241565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.241675 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:06.241855 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.242577 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.243907 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.244632 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.246477 (+ 1845us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.246585 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.246652 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.246741 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":89,"threads_started":1}
I20260806 22:46:06.246752 7450 tablet_service.cc:1467] Processing CreateTablet for tablet dfa9f957fbb544c7bac83542c5d5bc26 (DEFAULT_TABLE table=test-table [id=88c56e2e601642eb85fd21a3de673e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:06.247045 7309 tablet_service.cc:1467] Processing CreateTablet for tablet dfa9f957fbb544c7bac83542c5d5bc26 (DEFAULT_TABLE table=test-table [id=88c56e2e601642eb85fd21a3de673e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:06.247181 7450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfa9f957fbb544c7bac83542c5d5bc26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:06.247413 7309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfa9f957fbb544c7bac83542c5d5bc26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:06.247709 7164 tablet_service.cc:1467] Processing CreateTablet for tablet dfa9f957fbb544c7bac83542c5d5bc26 (DEFAULT_TABLE table=test-table [id=88c56e2e601642eb85fd21a3de673e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:06.248028 7164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfa9f957fbb544c7bac83542c5d5bc26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:06.251396 7537 tablet_bootstrap.cc:492] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f: Bootstrap starting.
I20260806 22:46:06.251725 7538 tablet_bootstrap.cc:492] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Bootstrap starting.
I20260806 22:46:06.252259 7537 tablet_bootstrap.cc:654] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:06.252496 7538 tablet_bootstrap.cc:654] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:06.253003 7537 log.cc:824] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:06.253096 7538 log.cc:824] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:06.253783 7536 tablet_bootstrap.cc:492] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: Bootstrap starting.
I20260806 22:46:06.253916 7537 tablet_bootstrap.cc:492] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f: No bootstrap required, opened a new log
I20260806 22:46:06.254026 7537 ts_tablet_manager.cc:1397] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:06.254124 7538 tablet_bootstrap.cc:492] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: No bootstrap required, opened a new log
I20260806 22:46:06.254194 7538 ts_tablet_manager.cc:1397] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:06.254462 7536 tablet_bootstrap.cc:654] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:06.255069 7536 log.cc:824] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:06.255740 7536 tablet_bootstrap.cc:492] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: No bootstrap required, opened a new log
I20260806 22:46:06.255795 7536 ts_tablet_manager.cc:1397] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:46:06.256129 7538 raft_consensus.cc:348] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.256326 7538 raft_consensus.cc:374] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:06.256203 7537 raft_consensus.cc:348] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.256407 7538 raft_consensus.cc:729] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145c4a8b2d38404dbfd3c9e9d4dc07a7, State: Initialized, Role: FOLLOWER
I20260806 22:46:06.256445 7537 raft_consensus.cc:374] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:06.256546 7538 consensus_queue.cc:260] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.256587 7537 raft_consensus.cc:729] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86198e2492354f27a354c2dd7ea1478f, State: Initialized, Role: FOLLOWER
I20260806 22:46:06.256834 7538 ts_tablet_manager.cc:1428] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:06.256852 7378 heartbeater.cc:503] Master 127.19.74.254:41501 was elected leader, sending a full tablet report...
I20260806 22:46:06.256873 7537 consensus_queue.cc:260] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.257233 7537 ts_tablet_manager.cc:1428] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:06.257268 7236 heartbeater.cc:503] Master 127.19.74.254:41501 was elected leader, sending a full tablet report...
I20260806 22:46:06.257959 7536 raft_consensus.cc:348] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.258142 7536 raft_consensus.cc:374] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:06.258184 7536 raft_consensus.cc:729] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77fcf4c6ef3b4307ba9cf82799d2a005, State: Initialized, Role: FOLLOWER
I20260806 22:46:06.258302 7536 consensus_queue.cc:260] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.258560 7536 ts_tablet_manager.cc:1428] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:46:06.258577 7519 heartbeater.cc:503] Master 127.19.74.254:41501 was elected leader, sending a full tablet report...
W20260806 22:46:06.313035 7379 tablet.cc:2369] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:06.321960 7544 raft_consensus.cc:482] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:06.322078 7544 raft_consensus.cc:504] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.322425 7544 leader_election.cc:288] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145c4a8b2d38404dbfd3c9e9d4dc07a7 (127.19.74.194:33589), 86198e2492354f27a354c2dd7ea1478f (127.19.74.193:36099)
I20260806 22:46:06.326352 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.322683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:43961 (local address 127.19.74.194:33589)
0806 22:46:06.322747 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.322749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.322763 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:46:06.322854 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.322854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.322881 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.322941 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:06.323155 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.323877 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.325403 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.325987 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.326063 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.326071 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.326245 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.326284 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:46:06.326352 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.322754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:52847 (local address 127.19.74.193:36099)
0806 22:46:06.322806 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.322808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.322912 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:46:06.322941 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.322942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.322953 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.323004 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:06.323154 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.323873 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.325403 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.325987 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.326063 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.326072 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.326245 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.326280 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:46:06.326380 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.322574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:33589 (local address 127.19.74.195:43961)
0806 22:46:06.322720 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.322732 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.322781 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.322966 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.322967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.322968 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:06.323059 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.323062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.323960 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.323961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.325305 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.325308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.326075 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.326076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.326157 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.326159 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.326223 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.326268 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":53}
I20260806 22:46:06.326393 7545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.322585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36099 (local address 127.19.74.195:52847)
0806 22:46:06.322818 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.322826 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.322876 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.323020 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.323021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.323022 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:06.323079 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.323080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.323960 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.323961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.325305 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.325308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.326075 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.326076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.326154 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.326159 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.326217 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.326268 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":108,"thread_start_us":151,"threads_started":1}
I20260806 22:46:06.326804 7330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfa9f957fbb544c7bac83542c5d5bc26" candidate_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" is_pre_election: true
I20260806 22:46:06.326995 7330 raft_consensus.cc:2454] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77fcf4c6ef3b4307ba9cf82799d2a005 in term 0.
I20260806 22:46:06.327122 7186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfa9f957fbb544c7bac83542c5d5bc26" candidate_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86198e2492354f27a354c2dd7ea1478f" is_pre_election: true
I20260806 22:46:06.327270 7403 leader_election.cc:302] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145c4a8b2d38404dbfd3c9e9d4dc07a7, 77fcf4c6ef3b4307ba9cf82799d2a005; no voters:
I20260806 22:46:06.327298 7186 raft_consensus.cc:2454] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77fcf4c6ef3b4307ba9cf82799d2a005 in term 0.
I20260806 22:46:06.327394 7544 raft_consensus.cc:2781] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:06.327468 7544 raft_consensus.cc:482] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:06.327494 7544 raft_consensus.cc:3037] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:06.328191 7544 raft_consensus.cc:504] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.328339 7544 leader_election.cc:288] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [CANDIDATE]: Term 1 election: Requested vote from peers 145c4a8b2d38404dbfd3c9e9d4dc07a7 (127.19.74.194:33589), 86198e2492354f27a354c2dd7ea1478f (127.19.74.193:36099)
I20260806 22:46:06.328506 7186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfa9f957fbb544c7bac83542c5d5bc26" candidate_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86198e2492354f27a354c2dd7ea1478f"
I20260806 22:46:06.328513 7330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfa9f957fbb544c7bac83542c5d5bc26" candidate_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7"
I20260806 22:46:06.328613 7330 raft_consensus.cc:3037] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:06.328621 7186 raft_consensus.cc:3037] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:06.329375 7186 raft_consensus.cc:2454] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77fcf4c6ef3b4307ba9cf82799d2a005 in term 1.
I20260806 22:46:06.329465 7330 raft_consensus.cc:2454] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77fcf4c6ef3b4307ba9cf82799d2a005 in term 1.
I20260806 22:46:06.329574 7402 leader_election.cc:302] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77fcf4c6ef3b4307ba9cf82799d2a005, 86198e2492354f27a354c2dd7ea1478f; no voters:
I20260806 22:46:06.329766 7544 raft_consensus.cc:2781] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:06.329932 7544 raft_consensus.cc:686] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 1 LEADER]: Becoming Leader. State: Replica: 77fcf4c6ef3b4307ba9cf82799d2a005, State: Running, Role: LEADER
I20260806 22:46:06.330049 7544 consensus_queue.cc:237] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } }
I20260806 22:46:06.331009 7030 catalog_manager.cc:5495] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77fcf4c6ef3b4307ba9cf82799d2a005 (127.19.74.195). New cstate: current_term: 1 leader_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77fcf4c6ef3b4307ba9cf82799d2a005" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 41149 } health_report { overall_health: HEALTHY } } }
I20260806 22:46:06.337009 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.334565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.19.74.195:41149)
0806 22:46:06.334647 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.334649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.334659 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:06.334744 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.334745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.334762 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.334846 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:06.334999 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.335458 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.336225 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.336443 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.336518 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.336679 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:06.336781 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:06.336866 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.336893 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.336913 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:46:06.338699 7330 raft_consensus.cc:1261] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 1 FOLLOWER]: Refusing update from remote peer 77fcf4c6ef3b4307ba9cf82799d2a005: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:06.338984 7186 raft_consensus.cc:1261] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 1 FOLLOWER]: Refusing update from remote peer 77fcf4c6ef3b4307ba9cf82799d2a005: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:06.339005 7544 consensus_queue.cc:1035] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145c4a8b2d38404dbfd3c9e9d4dc07a7" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:06.339262 7544 consensus_queue.cc:1035] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86198e2492354f27a354c2dd7ea1478f" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:06.341967 7552 mvcc.cc:205] Tried to move back new op lower bound from 7315686876520869888 to 7315686876488663040. Current Snapshot: MvccSnapshot[applied={T|T < 7315686876520869888 or (T in {7315686876520869888})}]
I20260806 22:46:06.343719 7025 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:06.343807 7025 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:06.344575 7025 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efb1a161f808426294b0a1338486b0bb: Sending DeleteTablet for 3 replicas of tablet dfa9f957fbb544c7bac83542c5d5bc26
I20260806 22:46:06.344946 7164 tablet_service.cc:1514] Processing DeleteTablet for tablet dfa9f957fbb544c7bac83542c5d5bc26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:06 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:45130
I20260806 22:46:06.344977 7309 tablet_service.cc:1514] Processing DeleteTablet for tablet dfa9f957fbb544c7bac83542c5d5bc26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:06 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:45338
I20260806 22:46:06.345185 7563 tablet_replica.cc:330] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f: stopping tablet replica
I20260806 22:46:06.345212 7450 tablet_service.cc:1514] Processing DeleteTablet for tablet dfa9f957fbb544c7bac83542c5d5bc26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:06 UTC) from {username='kudu', principal='kudu/127.19.74.254@KRBTEST.COM'} at 127.0.0.1:36854
I20260806 22:46:06.345281 7563 raft_consensus.cc:2229] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:06.345366 7563 raft_consensus.cc:2258] T dfa9f957fbb544c7bac83542c5d5bc26 P 86198e2492354f27a354c2dd7ea1478f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:06.345372 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 7096
I20260806 22:46:06.346187 7561 tablet_replica.cc:330] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: stopping tablet replica
I20260806 22:46:06.346285 7561 raft_consensus.cc:2229] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:06.346377 7561 raft_consensus.cc:2258] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:06.346853 7561 ts_tablet_manager.cc:1905] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:06.347182 7564 tablet_replica.cc:330] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: stopping tablet replica
I20260806 22:46:06.347317 7564 raft_consensus.cc:2229] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:06.347457 7564 raft_consensus.cc:2258] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:06.348138 7564 ts_tablet_manager.cc:1905] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:06.348538 7561 ts_tablet_manager.cc:1918] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:06.348623 7561 log.cc:1196] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/dfa9f957fbb544c7bac83542c5d5bc26
I20260806 22:46:06.348906 7561 ts_tablet_manager.cc:1939] T dfa9f957fbb544c7bac83542c5d5bc26 P 145c4a8b2d38404dbfd3c9e9d4dc07a7: Deleting consensus metadata
I20260806 22:46:06.349398 7017 catalog_manager.cc:4841] TS 145c4a8b2d38404dbfd3c9e9d4dc07a7 (127.19.74.194:33589): tablet dfa9f957fbb544c7bac83542c5d5bc26 (table test-table [id=88c56e2e601642eb85fd21a3de673e20]) successfully deleted
I20260806 22:46:06.350463 7564 ts_tablet_manager.cc:1918] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:06.350600 7564 log.cc:1196] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/dfa9f957fbb544c7bac83542c5d5bc26
I20260806 22:46:06.350975 7564 ts_tablet_manager.cc:1939] T dfa9f957fbb544c7bac83542c5d5bc26 P 77fcf4c6ef3b4307ba9cf82799d2a005: Deleting consensus metadata
I20260806 22:46:06.351395 7017 catalog_manager.cc:4841] TS 77fcf4c6ef3b4307ba9cf82799d2a005 (127.19.74.195:41149): tablet dfa9f957fbb544c7bac83542c5d5bc26 (table test-table [id=88c56e2e601642eb85fd21a3de673e20]) successfully deleted
W20260806 22:46:06.353679 7016 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:36099 (error 108)
I20260806 22:46:06.354190 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.354061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36099 (local address 127.0.0.1:45140)
0806 22:46:06.354117 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.354158 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:36099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:46:06.354283 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 7241
W20260806 22:46:06.356529 7016 catalog_manager.cc:4584] TS 86198e2492354f27a354c2dd7ea1478f (127.19.74.193:36099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dfa9f957fbb544c7bac83542c5d5bc26: Network error: Client connection negotiation failed: client connection to 127.19.74.193:36099: connect: Connection refused (error 111)
I20260806 22:46:06.360162 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 7383
I20260806 22:46:06.365353 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 6999
2026-08-06T22:46:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1951 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T22:46:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:06Z Disabled control of system clock
I20260806 22:46:06.394556 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37359
--webserver_interface=127.19.74.254
--webserver_port=0
--builtin_ntp_servers=127.19.74.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:37359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:06.473706 7570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:06.473946 7570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:06.473980 7570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:06.475134 7570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:06.475211 7570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:06.475232 7570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:06.475252 7570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:06.475283 7570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:06.476586 7570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:37359
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:37359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:06.476877 7570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:06.477162 7570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:06.481760 7581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:06.481853 7570 server_base.cc:1034] running on GCE node
W20260806 22:46:06.481781 7578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.481808 7576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:06.482268 7570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:06.482558 7570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:06.483736 7570 hybrid_clock.cc:648] HybridClock initialized: now 1786056366483715 us; error 37 us; skew 500 ppm
I20260806 22:46:06.484866 7570 webserver.cc:460] Webserver started at http://127.19.74.254:42325/ using document root <none> and password file <none>
I20260806 22:46:06.485090 7570 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:06.485157 7570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:06.485318 7570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:06.486335 7570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "0ae5f84c969444748ce5e2474149e63e"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "a73156781994f2a566f7f925f422e6f2"
server_key_iv: "bfc2d54f71ca2e8c5e17d38df390f418"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.486773 7570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "0ae5f84c969444748ce5e2474149e63e"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "a73156781994f2a566f7f925f422e6f2"
server_key_iv: "bfc2d54f71ca2e8c5e17d38df390f418"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.488035 7570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:06.488693 7587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.488857 7570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:06.488955 7570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "0ae5f84c969444748ce5e2474149e63e"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "a73156781994f2a566f7f925f422e6f2"
server_key_iv: "bfc2d54f71ca2e8c5e17d38df390f418"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.489037 7570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:06.517526 7570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:06.517879 7570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:06.518033 7570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:06.522318 7570 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:37359
I20260806 22:46:06.522352 7642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:37359 every 8 connection(s)
I20260806 22:46:06.522709 7570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:06.523725 7643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:06.525970 7643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e: Bootstrap starting.
I20260806 22:46:06.526533 7643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:06.526810 7643 log.cc:824] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:06.527437 7643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e: No bootstrap required, opened a new log
I20260806 22:46:06.529155 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 7570
I20260806 22:46:06.529054 7643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae5f84c969444748ce5e2474149e63e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37359 } }
I20260806 22:46:06.529232 7643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:06.529284 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:06.529284 7643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae5f84c969444748ce5e2474149e63e, State: Initialized, Role: FOLLOWER
I20260806 22:46:06.529435 7643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae5f84c969444748ce5e2474149e63e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37359 } }
I20260806 22:46:06.529490 19755 external_mini_cluster.cc:1428] Setting key 8d1b7c5233bed88f4cddd30fde08ccd8
I20260806 22:46:06.529522 7643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:06.529559 7643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:06.529619 7643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:06.530367 7643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae5f84c969444748ce5e2474149e63e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37359 } }
I20260806 22:46:06.530457 7643 leader_election.cc:302] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ae5f84c969444748ce5e2474149e63e; no voters:
I20260806 22:46:06.530665 7643 leader_election.cc:288] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:06.530719 7649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:06.530901 7649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [term 1 LEADER]: Becoming Leader. State: Replica: 0ae5f84c969444748ce5e2474149e63e, State: Running, Role: LEADER
I20260806 22:46:06.531008 7643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:06.531028 7649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae5f84c969444748ce5e2474149e63e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37359 } }
I20260806 22:46:06.531399 7649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ae5f84c969444748ce5e2474149e63e. Latest consensus state: current_term: 1 leader_uuid: "0ae5f84c969444748ce5e2474149e63e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae5f84c969444748ce5e2474149e63e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37359 } } }
I20260806 22:46:06.531410 7650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ae5f84c969444748ce5e2474149e63e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae5f84c969444748ce5e2474149e63e" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 37359 } } }
I20260806 22:46:06.531663 7650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:06.531869 7649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:06.532347 7658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:06.532958 7658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:06.534578 7658 catalog_manager.cc:1347] Generated new cluster ID: 93e5b9e48a96410b879fe9505fd5aa3f
I20260806 22:46:06.534698 7658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:06.536124 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.529981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34118 (local address 127.19.74.254:37359)
0806 22:46:06.530265 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.530274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.530306 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:46:06.530410 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.530416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.530520 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.530714 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:06.530949 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.532735 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.533804 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.534401 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.535067 (+ 666us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.535394 (+ 327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:06.535397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:06.535397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:06.535575 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:06.535610 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:06.535623 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:06.535680 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.535839 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.535917 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260806 22:46:06.547511 7658 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:06.548067 7658 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:06.553998 7658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e: Generated new TSK 0
I20260806 22:46:06.554181 7658 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:06.560544 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.193:0
--local_ip_for_outbound_sockets=127.19.74.193
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37359
--builtin_ntp_servers=127.19.74.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:06.645169 7668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:06.645386 7668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:06.645411 7668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:06.646744 7668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:06.646793 7668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:06.646819 7668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.193
I20260806 22:46:06.648303 7668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.74.193
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.193
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:06.648563 7668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:06.648872 7668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:06.654002 7674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.653965 7676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.654002 7673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:06.654471 7668 server_base.cc:1034] running on GCE node
I20260806 22:46:06.654709 7668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:06.654913 7668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:06.656049 7668 hybrid_clock.cc:648] HybridClock initialized: now 1786056366656042 us; error 33 us; skew 500 ppm
I20260806 22:46:06.657352 7668 webserver.cc:460] Webserver started at http://127.19.74.193:46015/ using document root <none> and password file <none>
I20260806 22:46:06.657604 7668 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:06.657653 7668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:06.657838 7668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:06.658987 7668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/data/instance:
uuid: "02af56896fb74de5a8b0edbcae836430"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "bf98e1f9910b66e26458bdf4cc6de7cf"
server_key_iv: "a3c0665f1c772b22b09d95d02037670b"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.659336 7668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance:
uuid: "02af56896fb74de5a8b0edbcae836430"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "bf98e1f9910b66e26458bdf4cc6de7cf"
server_key_iv: "a3c0665f1c772b22b09d95d02037670b"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.660696 7668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:06.661396 7682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.661574 7668 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:06.661680 7668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/wal
uuid: "02af56896fb74de5a8b0edbcae836430"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "bf98e1f9910b66e26458bdf4cc6de7cf"
server_key_iv: "a3c0665f1c772b22b09d95d02037670b"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.661752 7668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:06.683593 7668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:06.683943 7668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:06.684103 7668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:06.684367 7668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:06.684720 7668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:06.684772 7668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.684806 7668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:06.684845 7668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.691020 7668 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.193:36975
I20260806 22:46:06.691071 7802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.193:36975 every 8 connection(s)
I20260806 22:46:06.691397 7668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:06.696471 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 7668
I20260806 22:46:06.696575 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:06.696818 19755 external_mini_cluster.cc:1428] Setting key 95b2cbd3bb214cc84e7297dee647cde5
I20260806 22:46:06.696947 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.692118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:48677 (local address 127.19.74.254:37359)
0806 22:46:06.692184 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.692186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.692468 (+ 282us) server_negotiation.cc:407] Connection header received
0806 22:46:06.693081 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.693082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.693099 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.693158 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:06.693859 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.694347 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.695394 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.695624 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.695700 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.696598 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:06.696600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:06.696600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:06.696606 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:06.696628 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:06.696633 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:06.696717 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.696833 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.696866 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:46:06.697198 7805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.692114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37359 (local address 127.19.74.193:48677)
0806 22:46:06.692348 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.692382 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.692971 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.693232 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.693236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.693246 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:06.693751 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.693755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.694443 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.694444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.695296 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.695298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.695722 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.695723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.695856 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.696385 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:06.696386 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:06.696430 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:06.696462 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:06.696743 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:06.696750 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:06.696753 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.696833 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.696937 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260806 22:46:06.697781 7803 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37359
I20260806 22:46:06.697889 7803 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:06.698025 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.194:0
--local_ip_for_outbound_sockets=127.19.74.194
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37359
--builtin_ntp_servers=127.19.74.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:46:06.698091 7803 heartbeater.cc:511] Master 127.19.74.254:37359 requested a full tablet report, sending...
I20260806 22:46:06.698714 7603 ts_manager.cc:194] Registered new tserver with Master: 02af56896fb74de5a8b0edbcae836430 (127.19.74.193:36975)
I20260806 22:46:06.699653 7603 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.193:48677
W20260806 22:46:06.780243 7806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:06.780463 7806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:06.780498 7806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:06.781633 7806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:06.781697 7806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:06.781740 7806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.194
I20260806 22:46:06.783133 7806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.74.194
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.194
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:06.783392 7806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:06.783674 7806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:06.788779 7813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.788774 7816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.788782 7814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:06.788966 7806 server_base.cc:1034] running on GCE node
I20260806 22:46:06.789237 7806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:06.789479 7806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:06.790637 7806 hybrid_clock.cc:648] HybridClock initialized: now 1786056366790589 us; error 32 us; skew 500 ppm
I20260806 22:46:06.791936 7806 webserver.cc:460] Webserver started at http://127.19.74.194:40725/ using document root <none> and password file <none>
I20260806 22:46:06.792137 7806 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:06.792179 7806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:06.792304 7806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:06.793311 7806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/data/instance:
uuid: "f2a524a4fd1343e08eb2a09f8f116881"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "64602fa3c0221f603c28fcd30c7de3b4"
server_key_iv: "8b4c13a5ec5477a7e92c74b93cc6c878"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.793649 7806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance:
uuid: "f2a524a4fd1343e08eb2a09f8f116881"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "64602fa3c0221f603c28fcd30c7de3b4"
server_key_iv: "8b4c13a5ec5477a7e92c74b93cc6c878"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.794920 7806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:06.795615 7822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.795775 7806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:06.795862 7806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/wal
uuid: "f2a524a4fd1343e08eb2a09f8f116881"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "64602fa3c0221f603c28fcd30c7de3b4"
server_key_iv: "8b4c13a5ec5477a7e92c74b93cc6c878"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.795915 7806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:06.809222 7806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:06.809453 7806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:06.809545 7806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:06.809698 7806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:06.809935 7806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:06.809964 7806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.809983 7806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:06.809995 7806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.815553 7806 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.194:33783
I20260806 22:46:06.815641 7939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.194:33783 every 8 connection(s)
I20260806 22:46:06.815948 7806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:06.821275 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.816714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.194:37595 (local address 127.19.74.254:37359)
0806 22:46:06.816759 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.816761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.816987 (+ 226us) server_negotiation.cc:407] Connection header received
0806 22:46:06.817508 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.817510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.817524 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.817607 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:06.818283 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.818889 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.819856 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.820055 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.820136 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.820960 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:06.820961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:06.820962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:06.820967 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:06.820987 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:06.820991 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:06.821049 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.821169 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.821198 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:46:06.821485 7942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.816688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37359 (local address 127.19.74.194:37595)
0806 22:46:06.816878 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.816909 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.817393 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.817691 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.817695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.817712 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:06.818182 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.818186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.818986 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.818987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.819752 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.819754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.820148 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.820149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.820282 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.820810 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:06.820811 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:06.820838 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:06.820860 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:06.821051 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:06.821058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:06.821061 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.821169 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.821266 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260806 22:46:06.821952 7940 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37359
I20260806 22:46:06.822042 7940 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:06.822192 7940 heartbeater.cc:511] Master 127.19.74.254:37359 requested a full tablet report, sending...
I20260806 22:46:06.822520 7603 ts_manager.cc:194] Registered new tserver with Master: f2a524a4fd1343e08eb2a09f8f116881 (127.19.74.194:33783)
I20260806 22:46:06.822904 7603 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.194:37595
I20260806 22:46:06.823849 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 7806
I20260806 22:46:06.823956 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:06.824147 19755 external_mini_cluster.cc:1428] Setting key 4e4a0589ea08354a1602d6f92657c99e
I20260806 22:46:06.825217 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.74.195:0
--local_ip_for_outbound_sockets=127.19.74.195
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37359
--builtin_ntp_servers=127.19.74.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:46:06.908859 7943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:06.909087 7943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:06.909124 7943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:06.910342 7943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:06.910394 7943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:06.910435 7943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.74.195
I20260806 22:46:06.911828 7943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.74.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.74.195
--webserver_port=0
--tserver_master_addrs=127.19.74.254:37359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.74.195
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:06.912093 7943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:06.912413 7943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:06.917501 7951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.917531 7948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:06.917531 7949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:06.917740 7943 server_base.cc:1034] running on GCE node
I20260806 22:46:06.917986 7943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:06.918265 7943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:06.919415 7943 hybrid_clock.cc:648] HybridClock initialized: now 1786056366919404 us; error 26 us; skew 500 ppm
I20260806 22:46:06.920733 7943 webserver.cc:460] Webserver started at http://127.19.74.195:33293/ using document root <none> and password file <none>
I20260806 22:46:06.920941 7943 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:06.920981 7943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:06.921072 7943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:06.922045 7943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/data/instance:
uuid: "bc3f45f0f27449e19d663847086fca96"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "51a961c908d53ebebe54059c2d6b2293"
server_key_iv: "d27f3faea30ecf791741b33db9ec1e18"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.922369 7943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance:
uuid: "bc3f45f0f27449e19d663847086fca96"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "51a961c908d53ebebe54059c2d6b2293"
server_key_iv: "d27f3faea30ecf791741b33db9ec1e18"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.923715 7943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:06.924383 7957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.924548 7943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:06.924652 7943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/wal
uuid: "bc3f45f0f27449e19d663847086fca96"
format_stamp: "Formatted at 2026-08-06 22:46:06 on dist-test-slave-pgkr"
server_key: "51a961c908d53ebebe54059c2d6b2293"
server_key_iv: "d27f3faea30ecf791741b33db9ec1e18"
server_key_version: "encryptionkey@0"
I20260806 22:46:06.924734 7943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:06.939741 7943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:06.940014 7943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:06.940166 7943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:06.940372 7943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:06.940711 7943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:06.940770 7943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.940809 7943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:06.940837 7943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:06.946393 7943 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.195:37991
I20260806 22:46:06.946435 8077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.195:37991 every 8 connection(s)
I20260806 22:46:06.946769 7943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:06.951043 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 7943
I20260806 22:46:06.951160 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:06.951382 19755 external_mini_cluster.cc:1428] Setting key 7b834be322ff1494947e2fb6074108b9
I20260806 22:46:06.952117 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.947527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.195:39503 (local address 127.19.74.254:37359)
0806 22:46:06.947594 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.947596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.947859 (+ 263us) server_negotiation.cc:407] Connection header received
0806 22:46:06.948299 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.948300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.948329 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.948386 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:06.949024 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.949512 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.950432 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.950650 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.950713 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.951785 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:06.951786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:06.951786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:06.951791 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:06.951808 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:06.951813 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:06.951864 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.951965 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.952014 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:46:06.952347 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.947516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:37359 (local address 127.19.74.195:39503)
0806 22:46:06.947743 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.947775 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.948215 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.948454 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.948457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.948465 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:06.948950 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.948954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.949589 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.949589 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.950358 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.950359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.950873 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.950875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.951043 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.951614 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:46:06.951614 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:06.951642 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:46:06.951686 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:06.951877 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:06.951883 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:06.951886 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.951954 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.952057 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":129,"threads_started":1}
I20260806 22:46:06.952760 8078 heartbeater.cc:347] Connected to a master server at 127.19.74.254:37359
I20260806 22:46:06.952864 8078 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:06.953043 8078 heartbeater.cc:511] Master 127.19.74.254:37359 requested a full tablet report, sending...
I20260806 22:46:06.953399 7603 ts_manager.cc:194] Registered new tserver with Master: bc3f45f0f27449e19d663847086fca96 (127.19.74.195:37991)
I20260806 22:46:06.953734 7603 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.74.195:39503
I20260806 22:46:06.963382 19755 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:46:06.967494 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.964809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.19.74.254:37359)
0806 22:46:06.964877 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.964879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.964983 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:46:06.965022 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.965023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.965034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.965079 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:06.965224 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.965695 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.966357 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.966547 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.966595 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.967280 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:06.967281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:06.967281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:06.967285 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:06.967294 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:06.967297 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:06.967339 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.967382 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.967402 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:46:06.968889 7603 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:06.970301 7603 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:06.976432 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.972524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:33783 (local address 127.0.0.1:57642)
0806 22:46:06.972928 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.972938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.972979 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.973295 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.973296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.973298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:06.973336 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.973337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.974213 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.974214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.975343 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.975345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.976190 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.976191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.976246 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.976256 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.976309 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.976345 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":42,"thread_start_us":86,"threads_started":1}
I20260806 22:46:06.976469 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.972747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.19.74.194:33783)
0806 22:46:06.973089 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.973094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.973103 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:06.973141 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.973144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.973159 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.973229 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:06.973418 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.974122 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.975428 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.976079 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.976184 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.976273 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.976336 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.976379 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":85,"threads_started":1}
I20260806 22:46:06.976670 8086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.972349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:37991 (local address 127.0.0.1:51268)
0806 22:46:06.972606 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.972616 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.972659 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.972875 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.972878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.972879 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:06.972941 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.972943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.974241 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.974242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.975564 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.975567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.976440 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.976441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.976504 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.976508 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.976569 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.976613 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":35,"thread_start_us":94,"threads_started":1}
I20260806 22:46:06.976867 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.972452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.19.74.195:37991)
0806 22:46:06.972629 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.972634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.972650 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:46:06.972759 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.972762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.972776 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.972870 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:06.973021 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.974140 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.975656 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.976350 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.976426 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.976507 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.976733 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.976778 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260806 22:46:06.977496 7871 tablet_service.cc:1467] Processing CreateTablet for tablet df19c9c85073456581e3411f63d90df5 (DEFAULT_TABLE table=test-table [id=1e0da2eed547405292556a00bce231c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:06.977545 8008 tablet_service.cc:1467] Processing CreateTablet for tablet df19c9c85073456581e3411f63d90df5 (DEFAULT_TABLE table=test-table [id=1e0da2eed547405292556a00bce231c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:06.977751 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.972325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36975 (local address 127.0.0.1:54466)
0806 22:46:06.972515 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:06.972557 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:06.972644 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:06.972946 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:06.972946 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:06.972948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:06.972984 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.972985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.974303 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.974304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.975564 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:06.975567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.977503 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.977504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:06.977572 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.977576 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:06.977640 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:46:06.977679 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":171,"thread_start_us":60,"threads_started":1}
I20260806 22:46:06.977772 8088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:06.972420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54466 (local address 127.19.74.193:36975)
0806 22:46:06.972728 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:06.972732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:06.972742 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:06.972782 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:06.972784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:06.972801 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:06.972883 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:06.973144 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.974229 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.976588 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:06.977410 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:06.977487 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:06.977568 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:06.977653 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:46:06.977697 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":85,"threads_started":1}
I20260806 22:46:06.977888 7871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df19c9c85073456581e3411f63d90df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:06.977921 8008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df19c9c85073456581e3411f63d90df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:06.978438 7734 tablet_service.cc:1467] Processing CreateTablet for tablet df19c9c85073456581e3411f63d90df5 (DEFAULT_TABLE table=test-table [id=1e0da2eed547405292556a00bce231c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:06.978816 7734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df19c9c85073456581e3411f63d90df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:06.980665 8093 tablet_bootstrap.cc:492] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Bootstrap starting.
I20260806 22:46:06.981257 8093 tablet_bootstrap.cc:654] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:06.981297 8094 tablet_bootstrap.cc:492] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Bootstrap starting.
I20260806 22:46:06.981810 8093 log.cc:824] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:06.981936 8094 tablet_bootstrap.cc:654] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:06.982247 8094 log.cc:824] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:06.982717 8093 tablet_bootstrap.cc:492] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: No bootstrap required, opened a new log
I20260806 22:46:06.982796 8093 ts_tablet_manager.cc:1397] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:06.983050 8094 tablet_bootstrap.cc:492] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: No bootstrap required, opened a new log
I20260806 22:46:06.983103 8094 ts_tablet_manager.cc:1397] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:06.984619 8093 raft_consensus.cc:348] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:06.984773 8093 raft_consensus.cc:374] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:06.984820 8093 raft_consensus.cc:729] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a524a4fd1343e08eb2a09f8f116881, State: Initialized, Role: FOLLOWER
I20260806 22:46:06.984942 8093 consensus_queue.cc:260] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:06.984869 8094 raft_consensus.cc:348] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:06.985049 8094 raft_consensus.cc:374] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:06.985208 8094 raft_consensus.cc:729] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc3f45f0f27449e19d663847086fca96, State: Initialized, Role: FOLLOWER
I20260806 22:46:06.985224 8093 ts_tablet_manager.cc:1428] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:06.985351 7940 heartbeater.cc:503] Master 127.19.74.254:37359 was elected leader, sending a full tablet report...
I20260806 22:46:06.985384 8094 consensus_queue.cc:260] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:06.985723 8094 ts_tablet_manager.cc:1428] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:06.985742 8096 tablet_bootstrap.cc:492] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430: Bootstrap starting.
I20260806 22:46:06.985755 8078 heartbeater.cc:503] Master 127.19.74.254:37359 was elected leader, sending a full tablet report...
I20260806 22:46:06.986477 8096 tablet_bootstrap.cc:654] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:06.986879 8096 log.cc:824] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:06.987534 8096 tablet_bootstrap.cc:492] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430: No bootstrap required, opened a new log
I20260806 22:46:06.987587 8096 ts_tablet_manager.cc:1397] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:06.989328 8096 raft_consensus.cc:348] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:06.989502 8096 raft_consensus.cc:374] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:06.989552 8096 raft_consensus.cc:729] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02af56896fb74de5a8b0edbcae836430, State: Initialized, Role: FOLLOWER
I20260806 22:46:06.989681 8096 consensus_queue.cc:260] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:06.989918 8096 ts_tablet_manager.cc:1428] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:46:06.989955 7803 heartbeater.cc:503] Master 127.19.74.254:37359 was elected leader, sending a full tablet report...
W20260806 22:46:07.066521 7941 tablet.cc:2369] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:07.145418 8102 raft_consensus.cc:482] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:07.145556 8102 raft_consensus.cc:504] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:07.146001 8102 leader_election.cc:288] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc3f45f0f27449e19d663847086fca96 (127.19.74.195:37991), f2a524a4fd1343e08eb2a09f8f116881 (127.19.74.194:33783)
I20260806 22:46:07.150055 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:07.146297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:50847 (local address 127.19.74.195:37991)
0806 22:46:07.146343 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:07.146345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:07.146544 (+ 199us) server_negotiation.cc:407] Connection header received
0806 22:46:07.146584 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:07.146585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:07.146624 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:07.146701 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:07.146856 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.147555 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.149133 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.149675 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.149763 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:07.149771 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:07.149945 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:46:07.149984 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:46:07.150055 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:07.146302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.74.193:51427 (local address 127.19.74.194:33783)
0806 22:46:07.146357 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:07.146359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:07.146369 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:07.146402 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:07.146403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:07.146415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:07.146492 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:07.146726 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.147555 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.149132 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.149672 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.149763 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:07.149771 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:07.149945 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:46:07.149984 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:46:07.150068 7805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:07.146209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.194:33783 (local address 127.19.74.193:51427)
0806 22:46:07.146271 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:07.146283 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:07.146329 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:07.146523 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:07.146524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:07.146525 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:07.146636 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:07.146638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.147648 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.147649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:07.149034 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:07.149037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.149776 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.149777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:07.149858 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:07.149864 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:46:07.149920 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:46:07.149973 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":110}
I20260806 22:46:07.150074 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:07.146212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.195:37991 (local address 127.19.74.193:50847)
0806 22:46:07.146423 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:07.146431 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:07.146521 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:07.146725 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:07.146726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:07.146728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:07.146787 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:07.146788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.147648 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.147649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:07.149034 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:07.149037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.149776 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.149777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:07.149858 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:07.149864 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:46:07.149920 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:46:07.149971 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":87,"thread_start_us":102,"threads_started":1}
I20260806 22:46:07.150511 7891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df19c9c85073456581e3411f63d90df5" candidate_uuid: "02af56896fb74de5a8b0edbcae836430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a524a4fd1343e08eb2a09f8f116881" is_pre_election: true
I20260806 22:46:07.150727 7891 raft_consensus.cc:2454] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02af56896fb74de5a8b0edbcae836430 in term 0.
I20260806 22:46:07.150805 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df19c9c85073456581e3411f63d90df5" candidate_uuid: "02af56896fb74de5a8b0edbcae836430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc3f45f0f27449e19d663847086fca96" is_pre_election: true
I20260806 22:46:07.150959 8030 raft_consensus.cc:2454] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02af56896fb74de5a8b0edbcae836430 in term 0.
I20260806 22:46:07.150991 7685 leader_election.cc:302] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02af56896fb74de5a8b0edbcae836430, f2a524a4fd1343e08eb2a09f8f116881; no voters:
I20260806 22:46:07.151142 8102 raft_consensus.cc:2781] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:07.151196 8102 raft_consensus.cc:482] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:07.151217 8102 raft_consensus.cc:3037] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:07.151926 8102 raft_consensus.cc:504] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:07.152068 8102 leader_election.cc:288] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [CANDIDATE]: Term 1 election: Requested vote from peers bc3f45f0f27449e19d663847086fca96 (127.19.74.195:37991), f2a524a4fd1343e08eb2a09f8f116881 (127.19.74.194:33783)
I20260806 22:46:07.152253 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df19c9c85073456581e3411f63d90df5" candidate_uuid: "02af56896fb74de5a8b0edbcae836430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc3f45f0f27449e19d663847086fca96"
I20260806 22:46:07.152261 7891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df19c9c85073456581e3411f63d90df5" candidate_uuid: "02af56896fb74de5a8b0edbcae836430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a524a4fd1343e08eb2a09f8f116881"
I20260806 22:46:07.152364 8030 raft_consensus.cc:3037] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:07.152364 7891 raft_consensus.cc:3037] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:07.153265 7891 raft_consensus.cc:2454] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02af56896fb74de5a8b0edbcae836430 in term 1.
I20260806 22:46:07.153259 8030 raft_consensus.cc:2454] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02af56896fb74de5a8b0edbcae836430 in term 1.
I20260806 22:46:07.153460 7683 leader_election.cc:302] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02af56896fb74de5a8b0edbcae836430, bc3f45f0f27449e19d663847086fca96; no voters:
I20260806 22:46:07.153564 8102 raft_consensus.cc:2781] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:07.153738 8102 raft_consensus.cc:686] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 1 LEADER]: Becoming Leader. State: Replica: 02af56896fb74de5a8b0edbcae836430, State: Running, Role: LEADER
I20260806 22:46:07.153855 8102 consensus_queue.cc:237] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } }
I20260806 22:46:07.154731 7603 catalog_manager.cc:5495] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02af56896fb74de5a8b0edbcae836430 (127.19.74.193). New cstate: current_term: 1 leader_uuid: "02af56896fb74de5a8b0edbcae836430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02af56896fb74de5a8b0edbcae836430" member_type: VOTER last_known_addr { host: "127.19.74.193" port: 36975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 } health_report { overall_health: UNKNOWN } } }
W20260806 22:46:07.192149 7804 tablet.cc:2369] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:46:07.197325 8079 tablet.cc:2369] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:07.202966 8088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:07.200772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54470 (local address 127.19.74.193:36975)
0806 22:46:07.200847 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:07.200849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:07.200857 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:07.200892 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:07.200893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:07.200912 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:07.200975 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:07.201086 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.201555 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.202232 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:07.202406 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:07.202456 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:07.202572 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:07.202736 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:07.202800 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:07.202828 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:46:07.202848 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:46:07.204900 8030 raft_consensus.cc:1261] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 1 FOLLOWER]: Refusing update from remote peer 02af56896fb74de5a8b0edbcae836430: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:07.204924 7891 raft_consensus.cc:1261] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 1 FOLLOWER]: Refusing update from remote peer 02af56896fb74de5a8b0edbcae836430: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:07.205319 8104 consensus_queue.cc:1035] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc3f45f0f27449e19d663847086fca96" member_type: VOTER last_known_addr { host: "127.19.74.195" port: 37991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:07.205458 8102 consensus_queue.cc:1035] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2a524a4fd1343e08eb2a09f8f116881" member_type: VOTER last_known_addr { host: "127.19.74.194" port: 33783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:07.207894 8107 mvcc.cc:205] Tried to move back new op lower bound from 7315686880068300800 to 7315686879862824960. Current Snapshot: MvccSnapshot[applied={T|T < 7315686880068300800}]
I20260806 22:46:07.208020 8108 mvcc.cc:205] Tried to move back new op lower bound from 7315686880068300800 to 7315686879862824960. Current Snapshot: MvccSnapshot[applied={T|T < 7315686880068300800}]
I20260806 22:46:07.209833 8109 mvcc.cc:205] Tried to move back new op lower bound from 7315686880068300800 to 7315686879862824960. Current Snapshot: MvccSnapshot[applied={T|T < 7315686880068300800}]
I20260806 22:46:07.210166 7603 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:07.210264 7603 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:07.211086 7603 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ae5f84c969444748ce5e2474149e63e: Sending DeleteTablet for 3 replicas of tablet df19c9c85073456581e3411f63d90df5
I20260806 22:46:07.211447 8008 tablet_service.cc:1514] Processing DeleteTablet for tablet df19c9c85073456581e3411f63d90df5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:07 UTC) from {username='slave'} at 127.0.0.1:51268
I20260806 22:46:07.211489 7734 tablet_service.cc:1514] Processing DeleteTablet for tablet df19c9c85073456581e3411f63d90df5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:07 UTC) from {username='slave'} at 127.0.0.1:54466
I20260806 22:46:07.211674 7871 tablet_service.cc:1514] Processing DeleteTablet for tablet df19c9c85073456581e3411f63d90df5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:07 UTC) from {username='slave'} at 127.0.0.1:57642
I20260806 22:46:07.211709 8117 tablet_replica.cc:330] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: stopping tablet replica
I20260806 22:46:07.211804 8117 raft_consensus.cc:2229] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:07.211864 8119 tablet_replica.cc:330] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: stopping tablet replica
I20260806 22:46:07.211897 8117 raft_consensus.cc:2258] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:07.211957 8119 raft_consensus.cc:2229] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:07.211958 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 7668
I20260806 22:46:07.212040 8118 tablet_replica.cc:330] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430: stopping tablet replica
I20260806 22:46:07.212054 8119 raft_consensus.cc:2258] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:07.212118 8118 raft_consensus.cc:2229] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:07.212200 8118 raft_consensus.cc:2258] T df19c9c85073456581e3411f63d90df5 P 02af56896fb74de5a8b0edbcae836430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:07.213006 8119 ts_tablet_manager.cc:1905] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:07.213778 8117 ts_tablet_manager.cc:1905] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:07.214946 8119 ts_tablet_manager.cc:1918] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:07.215046 8119 log.cc:1196] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-1/wal/wals/df19c9c85073456581e3411f63d90df5
I20260806 22:46:07.215312 8119 ts_tablet_manager.cc:1939] T df19c9c85073456581e3411f63d90df5 P f2a524a4fd1343e08eb2a09f8f116881: Deleting consensus metadata
I20260806 22:46:07.215694 8117 ts_tablet_manager.cc:1918] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:07.215806 8117 log.cc:1196] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Deleting WAL directory at /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056127658350-19755-0/minicluster-data/ts-2/wal/wals/df19c9c85073456581e3411f63d90df5
I20260806 22:46:07.215963 7590 catalog_manager.cc:4841] TS f2a524a4fd1343e08eb2a09f8f116881 (127.19.74.194:33783): tablet df19c9c85073456581e3411f63d90df5 (table test-table [id=1e0da2eed547405292556a00bce231c7]) successfully deleted
I20260806 22:46:07.216078 8117 ts_tablet_manager.cc:1939] T df19c9c85073456581e3411f63d90df5 P bc3f45f0f27449e19d663847086fca96: Deleting consensus metadata
I20260806 22:46:07.216528 7588 catalog_manager.cc:4841] TS bc3f45f0f27449e19d663847086fca96 (127.19.74.195:37991): tablet df19c9c85073456581e3411f63d90df5 (table test-table [id=1e0da2eed547405292556a00bce231c7]) successfully deleted
W20260806 22:46:07.219813 7590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.74.193:36975 (error 108)
I20260806 22:46:07.220039 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 7806
I20260806 22:46:07.220943 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:07.220149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.193:36975 (local address 127.0.0.1:54480)
0806 22:46:07.220877 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:07.220917 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.193:36975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":707}
W20260806 22:46:07.227272 7590 catalog_manager.cc:4584] TS 02af56896fb74de5a8b0edbcae836430 (127.19.74.193:36975): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df19c9c85073456581e3411f63d90df5: Network error: Client connection negotiation failed: client connection to 127.19.74.193:36975: connect: Connection refused (error 111)
I20260806 22:46:07.229343 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 7943
I20260806 22:46:07.235041 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 7570
2026-08-06T22:46:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (869 ms)
[----------] 6 tests from EncryptionPolicyTest (8414 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:07 dist-test-slave-pgkr krb5kdc[8123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:07 dist-test-slave-pgkr krb5kdc[8123](info): set up 2 sockets
Aug 06 22:46:07 dist-test-slave-pgkr krb5kdc[8123](info): commencing operation
krb5kdc: starting...
W20260806 22:46:08.285224 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:08 dist-test-slave-pgkr krb5kdc[8123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:08.405580 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:08.489934 8136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:08.490155 8136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:08.490206 8136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:08.491367 8136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:08.491431 8136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:08.491456 8136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:08.491487 8136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:08.491513 8136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:08.493002 8136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:08.493290 8136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:08.493534 8136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:08.498407 8144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:08.498440 8142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:08.498409 8141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:08.498577 8136 server_base.cc:1034] running on GCE node
I20260806 22:46:08.498843 8136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:08.499157 8136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:08.500330 8136 hybrid_clock.cc:648] HybridClock initialized: now 1786056368500314 us; error 45 us; skew 500 ppm
Aug 06 22:46:08 dist-test-slave-pgkr krb5kdc[8123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056368, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:08.502830 8136 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:08.503271 8136 webserver.cc:460] Webserver started at http://127.0.0.1:36111/ using document root <none> and password file <none>
I20260806 22:46:08.503507 8136 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:08.503577 8136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:08.503743 8136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:08.504801 8136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "589681a965ad425787d85305809b6bd7"
format_stamp: "Formatted at 2026-08-06 22:46:08 on dist-test-slave-pgkr"
server_key: "d342ab625b8905aa1cf2d855f2c6a80c"
server_key_iv: "8d0f065e421fdfb7f5dd5fb82a4a932e"
server_key_version: "encryptionkey@0"
I20260806 22:46:08.505164 8136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "589681a965ad425787d85305809b6bd7"
format_stamp: "Formatted at 2026-08-06 22:46:08 on dist-test-slave-pgkr"
server_key: "d342ab625b8905aa1cf2d855f2c6a80c"
server_key_iv: "8d0f065e421fdfb7f5dd5fb82a4a932e"
server_key_version: "encryptionkey@0"
I20260806 22:46:08.506484 8136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:08.507267 8152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:08.507447 8136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:08.507550 8136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "589681a965ad425787d85305809b6bd7"
format_stamp: "Formatted at 2026-08-06 22:46:08 on dist-test-slave-pgkr"
server_key: "d342ab625b8905aa1cf2d855f2c6a80c"
server_key_iv: "8d0f065e421fdfb7f5dd5fb82a4a932e"
server_key_version: "encryptionkey@0"
I20260806 22:46:08.507647 8136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:08.522042 8136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:08.524487 8136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:08.524669 8136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:08.529102 8136 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39777
I20260806 22:46:08.529135 8206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39777 every 8 connection(s)
I20260806 22:46:08.529488 8136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:08.530337 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 8136
I20260806 22:46:08.530457 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:08.530474 8207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:08.530701 19755 external_mini_cluster.cc:1428] Setting key f968814871a32f8036d8f27fd8ec8226
I20260806 22:46:08.533730 8207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7: Bootstrap starting.
I20260806 22:46:08.534442 8207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:08.534770 8207 log.cc:824] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:08.535492 8207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7: No bootstrap required, opened a new log
I20260806 22:46:08.537307 8207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589681a965ad425787d85305809b6bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } }
I20260806 22:46:08.537467 8207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:08.537518 8207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589681a965ad425787d85305809b6bd7, State: Initialized, Role: FOLLOWER
Aug 06 22:46:08 dist-test-slave-pgkr krb5kdc[8123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:46:08.537673 8207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589681a965ad425787d85305809b6bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } }
I20260806 22:46:08.537793 8207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:08.537832 8207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:08.537878 8207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:08.538400 8207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589681a965ad425787d85305809b6bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } }
I20260806 22:46:08.538522 8207 leader_election.cc:302] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 589681a965ad425787d85305809b6bd7; no voters:
I20260806 22:46:08.538738 8207 leader_election.cc:288] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:08.538798 8212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:08.538949 8212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [term 1 LEADER]: Becoming Leader. State: Replica: 589681a965ad425787d85305809b6bd7, State: Running, Role: LEADER
I20260806 22:46:08.539001 8207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:08.539124 8212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589681a965ad425787d85305809b6bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } }
I20260806 22:46:08.540987 8214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 589681a965ad425787d85305809b6bd7. Latest consensus state: current_term: 1 leader_uuid: "589681a965ad425787d85305809b6bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589681a965ad425787d85305809b6bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } } }
I20260806 22:46:08.540983 8213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "589681a965ad425787d85305809b6bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589681a965ad425787d85305809b6bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } } }
W20260806 22:46:08.541078 8228 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 22:46:08.541256 8214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [sys.catalog]: This master's current role is: LEADER
W20260806 22:46:08.541347 8228 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:08.541265 8213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:08.541541 8230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:08.541728 8230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:08.542526 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:08.531285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49541 (local address 127.0.0.1:39777)
0806 22:46:08.531470 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:08.531479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:08.531514 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:46:08.531997 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:08.532003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:08.532051 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:08.532161 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:08.533028 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:08.533985 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:08.535043 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:08.535519 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:08.535670 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:08.538229 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:08.538233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:08.538233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:08.538252 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:08.540588 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:08.541176 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:08.541180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:08.541180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:08.541292 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:08.541582 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:08.541583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:08.541583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:08.541890 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:08.542034 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:08.542200 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:46:08.542313 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260806 22:46:08.543485 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:08.543514 8230 catalog_manager.cc:1347] Generated new cluster ID: b3f38604d91045099ec34847b4ac2373
I20260806 22:46:08.543561 8230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:08.561784 8230 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:08.562528 8230 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:08.562657 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:08.554934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39831 (local address 127.0.0.1:39777)
0806 22:46:08.554974 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:08.554976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:08.555066 (+ 90us) server_negotiation.cc:407] Connection header received
0806 22:46:08.555133 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:08.555134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:08.555143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:08.555190 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:08.555868 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:08.556587 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:08.557686 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:08.557977 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:08.558034 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:08.559576 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:08.559577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:08.559578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:08.559585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:08.561225 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:08.561715 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:08.561716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:08.561720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:08.561786 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:08.562025 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:08.562026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:08.562026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:08.562136 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:08.562248 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:08.562371 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:46:08.562458 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":22}
I20260806 22:46:08.571930 8230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 589681a965ad425787d85305809b6bd7: Generated new TSK 0
I20260806 22:46:08.572122 8230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:08.578081 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 8136
W20260806 22:46:08.585004 8121 connection.cc:535] client connection to 127.0.0.1:39777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39777): Connection reset by peer (error 104)
2026-08-06T22:46:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:08 dist-test-slave-pgkr krb5kdc[8239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:08 dist-test-slave-pgkr krb5kdc[8239](info): set up 2 sockets
Aug 06 22:46:08 dist-test-slave-pgkr krb5kdc[8239](info): commencing operation
krb5kdc: starting...
W20260806 22:46:09.632992 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:09 dist-test-slave-pgkr krb5kdc[8239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:09.749425 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41683
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:09.830025 8252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:09.830219 8252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:09.830240 8252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:09.831347 8252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:09.831382 8252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:09.831396 8252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:09.831408 8252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:09.831420 8252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:09.832819 8252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41683
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:09.833004 8252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:09.833168 8252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:09.837931 8258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:09.837916 8257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:09.837915 8260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:09.838183 8252 server_base.cc:1034] running on GCE node
I20260806 22:46:09.838416 8252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:09.838766 8252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:09.839905 8252 hybrid_clock.cc:648] HybridClock initialized: now 1786056369839894 us; error 27 us; skew 500 ppm
Aug 06 22:46:09 dist-test-slave-pgkr krb5kdc[8239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:09.845574 8252 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:09.846124 8252 webserver.cc:460] Webserver started at http://127.0.0.1:36493/ using document root <none> and password file <none>
I20260806 22:46:09.846410 8252 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:09.846458 8252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:09.846645 8252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:09.847740 8252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "61a17f36c2ce4bc1b80a22a300c800db"
format_stamp: "Formatted at 2026-08-06 22:46:09 on dist-test-slave-pgkr"
server_key: "a287b6fe2b80ca032a1ff5f240f2b724"
server_key_iv: "e9eb92b07566bf0262928267a1e8a1fa"
server_key_version: "encryptionkey@0"
I20260806 22:46:09.848131 8252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "61a17f36c2ce4bc1b80a22a300c800db"
format_stamp: "Formatted at 2026-08-06 22:46:09 on dist-test-slave-pgkr"
server_key: "a287b6fe2b80ca032a1ff5f240f2b724"
server_key_iv: "e9eb92b07566bf0262928267a1e8a1fa"
server_key_version: "encryptionkey@0"
I20260806 22:46:09.849577 8252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:09.850313 8268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:09.850479 8252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:09.850565 8252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "61a17f36c2ce4bc1b80a22a300c800db"
format_stamp: "Formatted at 2026-08-06 22:46:09 on dist-test-slave-pgkr"
server_key: "a287b6fe2b80ca032a1ff5f240f2b724"
server_key_iv: "e9eb92b07566bf0262928267a1e8a1fa"
server_key_version: "encryptionkey@0"
I20260806 22:46:09.850658 8252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:09.860107 8252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:09.861809 8252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:09.861913 8252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:09.865871 8252 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41683
I20260806 22:46:09.865942 8322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41683 every 8 connection(s)
I20260806 22:46:09.866258 8252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:09.867172 8323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:09.869395 8323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db: Bootstrap starting.
I20260806 22:46:09.869932 8323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:09.870227 8323 log.cc:824] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:09.870929 8323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db: No bootstrap required, opened a new log
I20260806 22:46:09.872495 8323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a17f36c2ce4bc1b80a22a300c800db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } }
I20260806 22:46:09.872653 8323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:09.872674 8323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a17f36c2ce4bc1b80a22a300c800db, State: Initialized, Role: FOLLOWER
I20260806 22:46:09.872812 8323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a17f36c2ce4bc1b80a22a300c800db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } }
I20260806 22:46:09.872889 8323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:09.872913 8323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:09.872946 8323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:09.873416 8323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a17f36c2ce4bc1b80a22a300c800db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } }
I20260806 22:46:09.873513 8323 leader_election.cc:302] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61a17f36c2ce4bc1b80a22a300c800db; no voters:
I20260806 22:46:09.873642 8323 leader_election.cc:288] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:09.873688 8326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:09.873842 8326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [term 1 LEADER]: Becoming Leader. State: Replica: 61a17f36c2ce4bc1b80a22a300c800db, State: Running, Role: LEADER
I20260806 22:46:09.873903 8323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:09.873907 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 8252
I20260806 22:46:09.873976 8326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a17f36c2ce4bc1b80a22a300c800db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } }
I20260806 22:46:09.874034 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:09.874243 19755 external_mini_cluster.cc:1428] Setting key 88ad9cd401aae0290035dfd86ad89d0e
I20260806 22:46:09.874661 8327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61a17f36c2ce4bc1b80a22a300c800db. Latest consensus state: current_term: 1 leader_uuid: "61a17f36c2ce4bc1b80a22a300c800db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a17f36c2ce4bc1b80a22a300c800db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } } }
I20260806 22:46:09.874835 8327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:09.875396 8328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61a17f36c2ce4bc1b80a22a300c800db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a17f36c2ce4bc1b80a22a300c800db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } } }
I20260806 22:46:09.875492 8328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db [sys.catalog]: This master's current role is: LEADER
W20260806 22:46:09.876268 8347 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:09.876333 8347 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:09.876403 8338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:09.876606 8338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:09.877974 8338 catalog_manager.cc:1347] Generated new cluster ID: 4ed8bc0835b94a8988ae8753ffe98986
I20260806 22:46:09.878026 8338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:09.880609 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:09.874892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36821 (local address 127.0.0.1:41683)
0806 22:46:09.875318 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:09.875327 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:09.875392 (+ 65us) server_negotiation.cc:407] Connection header received
0806 22:46:09.875551 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:09.875559 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:09.875622 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:09.875807 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:09.875999 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:09.877384 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:09.878474 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:09.879077 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:09.879177 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:09.879915 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:09.879918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:09.879919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:09.880124 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:09.880160 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:09.880173 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:09.880251 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:09.880380 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 22:46:09.880406 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":68,"threads_started":1}
I20260806 22:46:09.881541 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:09.886368 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:09.883125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50387 (local address 127.0.0.1:41683)
0806 22:46:09.883162 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:09.883164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:09.883182 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:46:09.883260 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:09.883261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:09.883271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:09.883340 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:09.883449 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:09.884163 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:09.885068 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:09.885374 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:09.885449 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:09.886099 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:09.886101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:09.886101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:09.886105 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:09.886127 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:09.886132 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:09.886196 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:09.886259 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:46:09.886272 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:09.890726 8338 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:09.891208 8338 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:09.899278 8338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61a17f36c2ce4bc1b80a22a300c800db: Generated new TSK 0
I20260806 22:46:09.899446 8338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:09.906226 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 8252
W20260806 22:46:09.912571 8237 connection.cc:535] client connection to 127.0.0.1:41683 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41683): Connection reset by peer (error 104)
2026-08-06T22:46:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:09 dist-test-slave-pgkr krb5kdc[8357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:09 dist-test-slave-pgkr krb5kdc[8357](info): set up 2 sockets
Aug 06 22:46:09 dist-test-slave-pgkr krb5kdc[8357](info): commencing operation
krb5kdc: starting...
W20260806 22:46:10.948344 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:11 dist-test-slave-pgkr krb5kdc[8357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:11.069216 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:11.152352 8370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:11.152582 8370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:11.152619 8370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:11.153738 8370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:11.153801 8370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:11.153825 8370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:11.153846 8370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:11.153874 8370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:11.155355 8370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39739
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:11.155637 8370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:11.155850 8370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:11.160557 8378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:11.160641 8375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:11.160720 8370 server_base.cc:1034] running on GCE node
W20260806 22:46:11.160558 8376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:11.161114 8370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:11.161407 8370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:11.162564 8370 hybrid_clock.cc:648] HybridClock initialized: now 1786056371162552 us; error 40 us; skew 500 ppm
Aug 06 22:46:11 dist-test-slave-pgkr krb5kdc[8357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056371, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:11.165091 8370 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:11.165539 8370 webserver.cc:460] Webserver started at http://127.0.0.1:42813/ using document root <none> and password file <none>
I20260806 22:46:11.165757 8370 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:11.165822 8370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:11.165964 8370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:11.167075 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "ea15f22b046f48b1b2b078cc593c32ab"
format_stamp: "Formatted at 2026-08-06 22:46:11 on dist-test-slave-pgkr"
server_key: "aa07ddd2b34a0bdd7fc1e6bdd4a74527"
server_key_iv: "0e4365fc3e201db42c2dbfedf1bf6554"
server_key_version: "encryptionkey@0"
I20260806 22:46:11.167430 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "ea15f22b046f48b1b2b078cc593c32ab"
format_stamp: "Formatted at 2026-08-06 22:46:11 on dist-test-slave-pgkr"
server_key: "aa07ddd2b34a0bdd7fc1e6bdd4a74527"
server_key_iv: "0e4365fc3e201db42c2dbfedf1bf6554"
server_key_version: "encryptionkey@0"
I20260806 22:46:11.168730 8370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:11.169404 8385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:11.169593 8370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:11.169715 8370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "ea15f22b046f48b1b2b078cc593c32ab"
format_stamp: "Formatted at 2026-08-06 22:46:11 on dist-test-slave-pgkr"
server_key: "aa07ddd2b34a0bdd7fc1e6bdd4a74527"
server_key_iv: "0e4365fc3e201db42c2dbfedf1bf6554"
server_key_version: "encryptionkey@0"
I20260806 22:46:11.169783 8370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:11.173103 8370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:11.174962 8370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:11.175117 8370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:11.179148 8370 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39739
I20260806 22:46:11.179177 8440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39739 every 8 connection(s)
I20260806 22:46:11.179513 8370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:11.180383 8441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:11.182554 8441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab: Bootstrap starting.
I20260806 22:46:11.183159 8441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:11.183389 8441 log.cc:824] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:11.183888 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 8370
I20260806 22:46:11.183998 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:11.184144 8441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab: No bootstrap required, opened a new log
I20260806 22:46:11.184191 19755 external_mini_cluster.cc:1428] Setting key 802df7f8996021f755ebcc97fe8d6f0d
I20260806 22:46:11.186571 8441 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea15f22b046f48b1b2b078cc593c32ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39739 } }
I20260806 22:46:11.186745 8441 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:11.186798 8441 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea15f22b046f48b1b2b078cc593c32ab, State: Initialized, Role: FOLLOWER
I20260806 22:46:11.186949 8441 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea15f22b046f48b1b2b078cc593c32ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39739 } }
I20260806 22:46:11.187011 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:11.184696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48295 (local address 127.0.0.1:39739)
0806 22:46:11.184885 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:11.184897 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:11.184934 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:46:11.185601 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:11.185610 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:11.185645 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:11.185794 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:11.186371 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:11.186375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:11.186375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:11.186380 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:11.186424 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:11.186440 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:11.186526 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:11.186565 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:46:11.186603 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260806 22:46:11.187060 8441 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:11.187101 8441 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:11.187157 8441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:11.187904 8441 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea15f22b046f48b1b2b078cc593c32ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39739 } }
I20260806 22:46:11.187973 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:11.188043 8441 leader_election.cc:302] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea15f22b046f48b1b2b078cc593c32ab; no voters:
I20260806 22:46:11.188206 8441 leader_election.cc:288] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:11.188248 8446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:11.188391 8446 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [term 1 LEADER]: Becoming Leader. State: Replica: ea15f22b046f48b1b2b078cc593c32ab, State: Running, Role: LEADER
I20260806 22:46:11.188460 8441 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:11.188499 8446 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea15f22b046f48b1b2b078cc593c32ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39739 } }
I20260806 22:46:11.188831 8446 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea15f22b046f48b1b2b078cc593c32ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea15f22b046f48b1b2b078cc593c32ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39739 } } }
I20260806 22:46:11.188905 8446 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:11.189059 8446 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea15f22b046f48b1b2b078cc593c32ab. Latest consensus state: current_term: 1 leader_uuid: "ea15f22b046f48b1b2b078cc593c32ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea15f22b046f48b1b2b078cc593c32ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39739 } } }
I20260806 22:46:11.189127 8446 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab [sys.catalog]: This master's current role is: LEADER
W20260806 22:46:11.190497 8467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:11.190554 8467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:11.190666 8454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:11.190914 8454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:11.191001 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:11.189562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48913 (local address 127.0.0.1:39739)
0806 22:46:11.189600 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:11.189602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:11.189752 (+ 150us) server_negotiation.cc:407] Connection header received
0806 22:46:11.189802 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:11.189803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:11.189807 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:11.190095 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:11.190465 (+ 370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:11.190466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:11.190466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:11.190471 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:11.190482 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:11.190486 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:11.190771 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:11.190924 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:46:11.190937 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:11.192441 8454 catalog_manager.cc:1347] Generated new cluster ID: 7c72988a09f94b3c8314dafe5f002534
I20260806 22:46:11.192493 8454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:11.214015 8454 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:11.214303 8454 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:11.221820 8454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea15f22b046f48b1b2b078cc593c32ab: Generated new TSK 0
I20260806 22:46:11.222029 8454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:11.226089 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 8370
2026-08-06T22:46:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:11 dist-test-slave-pgkr krb5kdc[8473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:11 dist-test-slave-pgkr krb5kdc[8473](info): set up 2 sockets
Aug 06 22:46:11 dist-test-slave-pgkr krb5kdc[8473](info): commencing operation
krb5kdc: starting...
W20260806 22:46:12.281571 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:12 dist-test-slave-pgkr krb5kdc[8473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:12.401325 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:12.481166 8487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:12.481375 8487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:12.481410 8487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:12.482448 8487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:12.482522 8487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:12.482548 8487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:12.482574 8487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:12.482594 8487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:12.483953 8487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:12.484215 8487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:12.484419 8487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:12.489005 8494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:12.489029 8497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:12.489074 8493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:12.489266 8487 server_base.cc:1034] running on GCE node
I20260806 22:46:12.489504 8487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:12.489797 8487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:12.490950 8487 hybrid_clock.cc:648] HybridClock initialized: now 1786056372490939 us; error 37 us; skew 500 ppm
Aug 06 22:46:12 dist-test-slave-pgkr krb5kdc[8473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056372, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:12.493247 8487 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:12.493707 8487 webserver.cc:460] Webserver started at http://127.0.0.1:34265/ using document root <none> and password file <none>
I20260806 22:46:12.493935 8487 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:12.493999 8487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:12.494138 8487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:12.495214 8487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "d9e5febe0bc549aa8e1c5010edc705a8"
format_stamp: "Formatted at 2026-08-06 22:46:12 on dist-test-slave-pgkr"
server_key: "ea6aa607c1a5ee44e3b74d30508721c9"
server_key_iv: "c013e41fcfdbc1f6b4711a1f06b06545"
server_key_version: "encryptionkey@0"
I20260806 22:46:12.495573 8487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "d9e5febe0bc549aa8e1c5010edc705a8"
format_stamp: "Formatted at 2026-08-06 22:46:12 on dist-test-slave-pgkr"
server_key: "ea6aa607c1a5ee44e3b74d30508721c9"
server_key_iv: "c013e41fcfdbc1f6b4711a1f06b06545"
server_key_version: "encryptionkey@0"
I20260806 22:46:12.496867 8487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:12.497603 8504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:12.497776 8487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:12.497880 8487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "d9e5febe0bc549aa8e1c5010edc705a8"
format_stamp: "Formatted at 2026-08-06 22:46:12 on dist-test-slave-pgkr"
server_key: "ea6aa607c1a5ee44e3b74d30508721c9"
server_key_iv: "c013e41fcfdbc1f6b4711a1f06b06545"
server_key_version: "encryptionkey@0"
I20260806 22:46:12.497948 8487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:12.509199 8487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:12.511077 8487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:12.511227 8487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:12.515229 8487 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40549
I20260806 22:46:12.515280 8557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40549 every 8 connection(s)
I20260806 22:46:12.515678 8487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:12.516536 8558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:12.518802 8558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8: Bootstrap starting.
I20260806 22:46:12.519361 8558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:12.519594 8558 log.cc:824] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:12.520224 8558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8: No bootstrap required, opened a new log
I20260806 22:46:12.521704 8558 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e5febe0bc549aa8e1c5010edc705a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } }
I20260806 22:46:12.521845 8558 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:12.521867 8558 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9e5febe0bc549aa8e1c5010edc705a8, State: Initialized, Role: FOLLOWER
I20260806 22:46:12.521991 8558 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e5febe0bc549aa8e1c5010edc705a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } }
I20260806 22:46:12.522069 8558 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:12.522092 8558 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:12.522127 8558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:12.522585 8558 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e5febe0bc549aa8e1c5010edc705a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } }
I20260806 22:46:12.522687 8558 leader_election.cc:302] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9e5febe0bc549aa8e1c5010edc705a8; no voters:
I20260806 22:46:12.522804 8558 leader_election.cc:288] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:12.522853 8561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:12.523008 8561 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [term 1 LEADER]: Becoming Leader. State: Replica: d9e5febe0bc549aa8e1c5010edc705a8, State: Running, Role: LEADER
I20260806 22:46:12.523068 8558 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:12.523160 8561 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e5febe0bc549aa8e1c5010edc705a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } }
I20260806 22:46:12.523553 8563 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9e5febe0bc549aa8e1c5010edc705a8. Latest consensus state: current_term: 1 leader_uuid: "d9e5febe0bc549aa8e1c5010edc705a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e5febe0bc549aa8e1c5010edc705a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } } }
I20260806 22:46:12.523636 8563 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:12.523695 8562 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9e5febe0bc549aa8e1c5010edc705a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e5febe0bc549aa8e1c5010edc705a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } } }
I20260806 22:46:12.523756 8562 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:12.523980 8572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:12.524569 8572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:12.525735 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 8487
I20260806 22:46:12.525846 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:12.526046 19755 external_mini_cluster.cc:1428] Setting key c0408c2deb8fc46ec99d671a7aad0be3
I20260806 22:46:12.526079 8572 catalog_manager.cc:1347] Generated new cluster ID: 24bd53b9e5b34f23addeff39979c7e5a
I20260806 22:46:12.526127 8572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:46:12 dist-test-slave-pgkr krb5kdc[8473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:46:12.539757 8584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:12.526697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56741 (local address 127.0.0.1:40549)
0806 22:46:12.530783 (+ 4086us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:12.530793 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:12.530841 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:46:12.530930 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:12.530935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:12.530985 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:12.531128 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:12.531793 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:12.532974 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:12.533750 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:12.534168 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:12.534279 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:12.536486 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:12.536490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:12.536491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:12.536503 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:12.538153 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:12.538525 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:12.538529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:12.538529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:12.538656 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:12.538874 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:12.538876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:12.538876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:12.539129 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:12.539286 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:12.539399 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:46:12.539538 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4032,"thread_start_us":79,"threads_started":1}
I20260806 22:46:12.540757 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:12.543926 8572 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:12.544641 8572 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:12.548513 8572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9e5febe0bc549aa8e1c5010edc705a8: Generated new TSK 0
I20260806 22:46:12.548781 8572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:12.548909 8584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:12.542083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37761 (local address 127.0.0.1:40549)
0806 22:46:12.542127 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:12.542128 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:12.542813 (+ 685us) server_negotiation.cc:407] Connection header received
0806 22:46:12.542884 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:12.542885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:12.542895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:12.542949 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:12.543467 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:12.544058 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:12.544718 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:12.544944 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:12.545013 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:12.546149 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:12.546151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:12.546151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:12.546158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:12.547566 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:12.547966 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:12.547967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:12.547968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:12.548020 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:12.548344 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:12.548345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:12.548345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:12.548480 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:12.548581 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:12.548688 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:46:12.548771 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":18}
I20260806 22:46:12.550661 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 8487
2026-08-06T22:46:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:12 dist-test-slave-pgkr krb5kdc[8592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:12 dist-test-slave-pgkr krb5kdc[8592](info): set up 2 sockets
Aug 06 22:46:12 dist-test-slave-pgkr krb5kdc[8592](info): commencing operation
krb5kdc: starting...
W20260806 22:46:13.601002 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:13 dist-test-slave-pgkr krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:13.720836 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:13.802188 8605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:13.802431 8605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:13.802464 8605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:13.803573 8605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:13.803637 8605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:13.803658 8605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:13.803675 8605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:13.803704 8605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:13.805124 8605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:13.805383 8605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:13.805599 8605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:46:13.810318 8605 server_base.cc:1034] running on GCE node
W20260806 22:46:13.810220 8611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:13.810209 8610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:13.810199 8613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:13.810763 8605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:13.810999 8605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:13.812136 8605 hybrid_clock.cc:648] HybridClock initialized: now 1786056373812124 us; error 28 us; skew 500 ppm
Aug 06 22:46:13 dist-test-slave-pgkr krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:13.814432 8605 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:13.814924 8605 webserver.cc:460] Webserver started at http://127.0.0.1:40733/ using document root <none> and password file <none>
I20260806 22:46:13.815161 8605 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:13.815222 8605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:13.815366 8605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:13.816362 8605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "70370c58721e44dc9ec8a903c4f6c75e"
format_stamp: "Formatted at 2026-08-06 22:46:13 on dist-test-slave-pgkr"
server_key: "2dc4a37fb811a23412d1725112d319ab"
server_key_iv: "539938cfc41aaca2d32ecac9117d67ae"
server_key_version: "encryptionkey@0"
I20260806 22:46:13.816721 8605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "70370c58721e44dc9ec8a903c4f6c75e"
format_stamp: "Formatted at 2026-08-06 22:46:13 on dist-test-slave-pgkr"
server_key: "2dc4a37fb811a23412d1725112d319ab"
server_key_iv: "539938cfc41aaca2d32ecac9117d67ae"
server_key_version: "encryptionkey@0"
I20260806 22:46:13.818012 8605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:13.818753 8620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:13.818921 8605 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:13.819020 8605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "70370c58721e44dc9ec8a903c4f6c75e"
format_stamp: "Formatted at 2026-08-06 22:46:13 on dist-test-slave-pgkr"
server_key: "2dc4a37fb811a23412d1725112d319ab"
server_key_iv: "539938cfc41aaca2d32ecac9117d67ae"
server_key_version: "encryptionkey@0"
I20260806 22:46:13.819113 8605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:13.844034 8605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:13.846946 8605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:13.847143 8605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:13.852231 8605 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43149
I20260806 22:46:13.852315 8674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43149 every 8 connection(s)
I20260806 22:46:13.852707 8605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:13.853870 8675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:13.855443 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 8605
I20260806 22:46:13.855571 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:13.855818 19755 external_mini_cluster.cc:1428] Setting key 07ee8955923b881e38fb587b38f93381
I20260806 22:46:13.857816 8675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e: Bootstrap starting.
I20260806 22:46:13.858783 8675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:13.859174 8675 log.cc:824] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:13.860028 8675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e: No bootstrap required, opened a new log
I20260806 22:46:13.862299 8678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:13.856360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35979 (local address 127.0.0.1:43149)
0806 22:46:13.856530 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:13.856540 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:13.856573 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:46:13.857447 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:13.857458 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:13.857540 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:13.857711 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:13.857983 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:13.859235 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:13.860443 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:13.860869 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:13.861005 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:13.861694 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:13.861699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:13.861699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:13.861867 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:13.861898 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:13.861908 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:13.861980 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:13.862058 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:46:13.862123 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260806 22:46:13.863258 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:13.863152 8675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70370c58721e44dc9ec8a903c4f6c75e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } }
I20260806 22:46:13.863328 8675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:13.863420 8675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70370c58721e44dc9ec8a903c4f6c75e, State: Initialized, Role: FOLLOWER
I20260806 22:46:13.863592 8675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70370c58721e44dc9ec8a903c4f6c75e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } }
I20260806 22:46:13.863721 8675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:13.863795 8675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:13.863883 8675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:13.864749 8675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70370c58721e44dc9ec8a903c4f6c75e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } }
I20260806 22:46:13.864878 8675 leader_election.cc:302] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70370c58721e44dc9ec8a903c4f6c75e; no voters:
I20260806 22:46:13.865067 8675 leader_election.cc:288] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:13.865116 8685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:13.865350 8685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [term 1 LEADER]: Becoming Leader. State: Replica: 70370c58721e44dc9ec8a903c4f6c75e, State: Running, Role: LEADER
I20260806 22:46:13.865417 8675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:13.865491 8685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70370c58721e44dc9ec8a903c4f6c75e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } }
I20260806 22:46:13.865926 8685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70370c58721e44dc9ec8a903c4f6c75e. Latest consensus state: current_term: 1 leader_uuid: "70370c58721e44dc9ec8a903c4f6c75e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70370c58721e44dc9ec8a903c4f6c75e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } } }
I20260806 22:46:13.866160 8685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:13.866072 8686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70370c58721e44dc9ec8a903c4f6c75e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70370c58721e44dc9ec8a903c4f6c75e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } } }
I20260806 22:46:13.866559 8686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:13.866825 8695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:13.867369 8695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:13.868387 8678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:13.864756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37077 (local address 127.0.0.1:43149)
0806 22:46:13.864792 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:13.864794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:13.864915 (+ 121us) server_negotiation.cc:407] Connection header received
0806 22:46:13.865014 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:13.865015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:13.865025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:13.865084 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:13.865289 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:13.865988 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:13.867009 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:13.867300 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:13.867345 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:13.868122 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:13.868123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:13.868124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:13.868129 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:13.868145 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:13.868149 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:13.868200 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:13.868278 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:46:13.868298 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:13.869155 8695 catalog_manager.cc:1347] Generated new cluster ID: 648ffde28b7044259964fed31742ff62
I20260806 22:46:13.869220 8695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:13.876324 8695 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:13.876801 8695 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:13.883127 8695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70370c58721e44dc9ec8a903c4f6c75e: Generated new TSK 0
I20260806 22:46:13.883298 8695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:13.886998 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 8605
2026-08-06T22:46:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:13 dist-test-slave-pgkr krb5kdc[8708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:13 dist-test-slave-pgkr krb5kdc[8708](info): set up 2 sockets
Aug 06 22:46:13 dist-test-slave-pgkr krb5kdc[8708](info): commencing operation
krb5kdc: starting...
W20260806 22:46:14.933861 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:15 dist-test-slave-pgkr krb5kdc[8708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:15.052809 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:15.134882 8722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:15.135073 8722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:15.135095 8722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:15.136137 8722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:15.136171 8722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:15.136185 8722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:15.136198 8722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:15.136209 8722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:15.137614 8722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:15.137800 8722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:15.137964 8722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:15.142593 8727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:15.142709 8722 server_base.cc:1034] running on GCE node
W20260806 22:46:15.142679 8730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:15.142692 8728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:15.143124 8722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:15.143419 8722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:15.144585 8722 hybrid_clock.cc:648] HybridClock initialized: now 1786056375144561 us; error 34 us; skew 500 ppm
Aug 06 22:46:15 dist-test-slave-pgkr krb5kdc[8708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:15.151145 8722 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:15.151629 8722 webserver.cc:460] Webserver started at http://127.0.0.1:40859/ using document root <none> and password file <none>
I20260806 22:46:15.151863 8722 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:15.151929 8722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:15.152082 8722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:15.153185 8722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "812f6a1da9bb47a1891106c54ef960bc"
format_stamp: "Formatted at 2026-08-06 22:46:15 on dist-test-slave-pgkr"
server_key: "eb3736a6fab01dbbf5fe0ada67bd5993"
server_key_iv: "ff87ea3bc440885398fe3d836166fdac"
server_key_version: "encryptionkey@0"
I20260806 22:46:15.153573 8722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "812f6a1da9bb47a1891106c54ef960bc"
format_stamp: "Formatted at 2026-08-06 22:46:15 on dist-test-slave-pgkr"
server_key: "eb3736a6fab01dbbf5fe0ada67bd5993"
server_key_iv: "ff87ea3bc440885398fe3d836166fdac"
server_key_version: "encryptionkey@0"
I20260806 22:46:15.155064 8722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:15.155864 8737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:15.156051 8722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:15.156163 8722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "812f6a1da9bb47a1891106c54ef960bc"
format_stamp: "Formatted at 2026-08-06 22:46:15 on dist-test-slave-pgkr"
server_key: "eb3736a6fab01dbbf5fe0ada67bd5993"
server_key_iv: "ff87ea3bc440885398fe3d836166fdac"
server_key_version: "encryptionkey@0"
I20260806 22:46:15.156236 8722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:15.159898 8722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:15.161741 8722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:15.161898 8722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:15.166200 8722 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43983
I20260806 22:46:15.166227 8790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43983 every 8 connection(s)
I20260806 22:46:15.166568 8722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:15.167443 8791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:15.167816 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 8722
I20260806 22:46:15.167943 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:15.168143 19755 external_mini_cluster.cc:1428] Setting key c11d1c8cd09a3791dfd420f04d9773b9
I20260806 22:46:15.170468 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:15.168674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46555 (local address 127.0.0.1:43983)
0806 22:46:15.168874 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:15.168889 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:15.168914 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:46:15.169303 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:15.169309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:15.169332 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:15.169448 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:15.170101 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:15.170104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:15.170104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:15.170108 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:15.170147 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:15.170158 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:15.170220 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:15.170313 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:46:15.170345 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260806 22:46:15.170514 8791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc: Bootstrap starting.
I20260806 22:46:15.171293 8791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:15.171489 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:15.171624 8791 log.cc:824] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:15.172425 8791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc: No bootstrap required, opened a new log
I20260806 22:46:15.174190 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:15.173057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40051 (local address 127.0.0.1:43983)
0806 22:46:15.173097 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:15.173099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:15.173191 (+ 92us) server_negotiation.cc:407] Connection header received
0806 22:46:15.173238 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:15.173238 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:15.173242 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:15.173293 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:15.173808 (+ 515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:15.173810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:15.173810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:15.173814 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:15.173826 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:15.173830 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:15.173885 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:15.173965 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:46:15.173975 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:46:15.174966 8791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f6a1da9bb47a1891106c54ef960bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43983 } }
I20260806 22:46:15.175096 8791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:15.175119 8791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 812f6a1da9bb47a1891106c54ef960bc, State: Initialized, Role: FOLLOWER
I20260806 22:46:15.175257 8791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f6a1da9bb47a1891106c54ef960bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43983 } }
I20260806 22:46:15.175349 8791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:15.175403 8791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:15.175455 8791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:15.175976 8791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f6a1da9bb47a1891106c54ef960bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43983 } }
I20260806 22:46:15.176090 8791 leader_election.cc:302] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 812f6a1da9bb47a1891106c54ef960bc; no voters:
I20260806 22:46:15.176246 8791 leader_election.cc:288] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:15.176283 8801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:15.176486 8801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [term 1 LEADER]: Becoming Leader. State: Replica: 812f6a1da9bb47a1891106c54ef960bc, State: Running, Role: LEADER
I20260806 22:46:15.176582 8801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f6a1da9bb47a1891106c54ef960bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43983 } }
I20260806 22:46:15.176658 8791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:15.176970 8802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "812f6a1da9bb47a1891106c54ef960bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f6a1da9bb47a1891106c54ef960bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43983 } } }
I20260806 22:46:15.176975 8803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 812f6a1da9bb47a1891106c54ef960bc. Latest consensus state: current_term: 1 leader_uuid: "812f6a1da9bb47a1891106c54ef960bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f6a1da9bb47a1891106c54ef960bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43983 } } }
I20260806 22:46:15.177150 8802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:15.177173 8803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:15.177556 8807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:15.178061 8807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:15.179590 8807 catalog_manager.cc:1347] Generated new cluster ID: a7ade8f2de2a45cdb2429196b3e89b90
I20260806 22:46:15.179651 8807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:15.203500 8807 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:15.203692 8807 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:15.207062 8807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 812f6a1da9bb47a1891106c54ef960bc: Generated new TSK 0
I20260806 22:46:15.207280 8807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:15.210676 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 8722
2026-08-06T22:46:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:15 dist-test-slave-pgkr krb5kdc[8822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:15 dist-test-slave-pgkr krb5kdc[8822](info): set up 2 sockets
Aug 06 22:46:15 dist-test-slave-pgkr krb5kdc[8822](info): commencing operation
krb5kdc: starting...
W20260806 22:46:16.257187 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:16 dist-test-slave-pgkr krb5kdc[8822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:16.377381 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:16.457037 8835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:16.457231 8835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:16.457252 8835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:16.458321 8835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:16.458355 8835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:16.458369 8835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:16.458382 8835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:16.458395 8835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:16.459743 8835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:16.459918 8835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:16.460114 8835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:46:16.465039 8835 server_base.cc:1034] running on GCE node
W20260806 22:46:16.464977 8841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:16.464989 8842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:16.465000 8845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:16.465508 8835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:16.465794 8835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:16.466943 8835 hybrid_clock.cc:648] HybridClock initialized: now 1786056376466932 us; error 29 us; skew 500 ppm
Aug 06 22:46:16 dist-test-slave-pgkr krb5kdc[8822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:16.469316 8835 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:16.469749 8835 webserver.cc:460] Webserver started at http://127.0.0.1:41717/ using document root <none> and password file <none>
I20260806 22:46:16.469967 8835 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:16.470046 8835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:16.470187 8835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:16.471264 8835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "38ce3442f29d4b16b742803b4e9c5e7b"
format_stamp: "Formatted at 2026-08-06 22:46:16 on dist-test-slave-pgkr"
server_key: "868382fae0c7829079cdb03e6cae63a5"
server_key_iv: "83d75674b49aa309f9ed844a7a68e5e7"
server_key_version: "encryptionkey@0"
I20260806 22:46:16.471616 8835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "38ce3442f29d4b16b742803b4e9c5e7b"
format_stamp: "Formatted at 2026-08-06 22:46:16 on dist-test-slave-pgkr"
server_key: "868382fae0c7829079cdb03e6cae63a5"
server_key_iv: "83d75674b49aa309f9ed844a7a68e5e7"
server_key_version: "encryptionkey@0"
I20260806 22:46:16.472983 8835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:16.473716 8853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:16.473872 8835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:16.473969 8835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "38ce3442f29d4b16b742803b4e9c5e7b"
format_stamp: "Formatted at 2026-08-06 22:46:16 on dist-test-slave-pgkr"
server_key: "868382fae0c7829079cdb03e6cae63a5"
server_key_iv: "83d75674b49aa309f9ed844a7a68e5e7"
server_key_version: "encryptionkey@0"
I20260806 22:46:16.474045 8835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:16.492959 8835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:16.494781 8835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:16.494946 8835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:16.498790 8835 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41231
I20260806 22:46:16.498862 8910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41231 every 8 connection(s)
I20260806 22:46:16.499224 8835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:16.500105 8911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:16.502274 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 8835
I20260806 22:46:16.502368 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:16.502409 8911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b: Bootstrap starting.
I20260806 22:46:16.502580 19755 external_mini_cluster.cc:1428] Setting key aca9a8d0caeda8ba53e79a144684498f
I20260806 22:46:16.503199 8911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:16.503633 8911 log.cc:824] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:16.504400 8911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b: No bootstrap required, opened a new log
I20260806 22:46:16.506717 8911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38ce3442f29d4b16b742803b4e9c5e7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41231 } }
I20260806 22:46:16.506868 8911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:16.506888 8911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ce3442f29d4b16b742803b4e9c5e7b, State: Initialized, Role: FOLLOWER
I20260806 22:46:16.507016 8911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38ce3442f29d4b16b742803b4e9c5e7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41231 } }
I20260806 22:46:16.507105 8911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:16.507136 8911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:16.507179 8911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:16.507913 8911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38ce3442f29d4b16b742803b4e9c5e7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41231 } }
I20260806 22:46:16.508028 8911 leader_election.cc:302] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38ce3442f29d4b16b742803b4e9c5e7b; no voters:
I20260806 22:46:16.508178 8911 leader_election.cc:288] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:16.508327 8918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:16.508426 8911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:16.508569 8918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [term 1 LEADER]: Becoming Leader. State: Replica: 38ce3442f29d4b16b742803b4e9c5e7b, State: Running, Role: LEADER
I20260806 22:46:16.508752 8918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38ce3442f29d4b16b742803b4e9c5e7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41231 } }
I20260806 22:46:16.509142 8916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38ce3442f29d4b16b742803b4e9c5e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38ce3442f29d4b16b742803b4e9c5e7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41231 } } }
I20260806 22:46:16.509214 8916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:16.509181 8919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38ce3442f29d4b16b742803b4e9c5e7b. Latest consensus state: current_term: 1 leader_uuid: "38ce3442f29d4b16b742803b4e9c5e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38ce3442f29d4b16b742803b4e9c5e7b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41231 } } }
I20260806 22:46:16.509271 8919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:16.509490 8927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 22:46:16 dist-test-slave-pgkr krb5kdc[8822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:46:16.510576 8927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:16.513441 8927 catalog_manager.cc:1347] Generated new cluster ID: e98963c2fc5948d68fad80ce10b185f6
I20260806 22:46:16.513499 8927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:16.514596 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:16.503145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:56139 (local address 127.0.0.1:41231)
0806 22:46:16.503331 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:16.503340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:16.503375 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:46:16.503975 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:16.503983 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:16.504067 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:16.504225 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:16.504852 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:16.506274 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:16.507074 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:16.507683 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:16.507802 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:16.510775 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:16.510779 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:16.510780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:16.510793 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:16.512884 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:16.513273 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:16.513276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:16.513276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:16.513381 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:16.513639 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:16.513640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:16.513640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:16.513941 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:16.514104 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:16.514253 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:46:16.514360 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260806 22:46:16.515551 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:16.523586 8927 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:16.524283 8927 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:16.524358 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:16.516945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:37235 (local address 127.0.0.1:41231)
0806 22:46:16.516978 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:16.516980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:16.517263 (+ 283us) server_negotiation.cc:407] Connection header received
0806 22:46:16.517371 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:16.517372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:16.517383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:16.517415 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:16.518162 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:16.518930 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:16.519874 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:16.520203 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:16.520273 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:16.521456 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:16.521457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:16.521458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:16.521464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:16.523042 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:16.523498 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:16.523499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:16.523499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:16.523548 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:16.523782 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:16.523783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:16.523783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:16.523880 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:16.523987 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:16.524103 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:46:16.524223 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":17}
I20260806 22:46:16.529165 8927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38ce3442f29d4b16b742803b4e9c5e7b: Generated new TSK 0
I20260806 22:46:16.529321 8927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:16.535199 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 8835
W20260806 22:46:16.541644 8820 connection.cc:535] client connection to 127.0.0.1:41231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41231): Connection reset by peer (error 104)
2026-08-06T22:46:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:16 dist-test-slave-pgkr krb5kdc[8945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:16 dist-test-slave-pgkr krb5kdc[8945](info): set up 2 sockets
Aug 06 22:46:16 dist-test-slave-pgkr krb5kdc[8945](info): commencing operation
krb5kdc: starting...
W20260806 22:46:17.573362 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:17 dist-test-slave-pgkr krb5kdc[8945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:17.689932 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38901
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:17.771811 8960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:17.772058 8960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:17.772109 8960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:17.773254 8960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:17.773325 8960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:17.773353 8960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:17.773386 8960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:17.773415 8960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:17.774904 8960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:17.775166 8960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:17.775395 8960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:17.780223 8965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:17.780232 8966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:17.780212 8968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:17.780516 8960 server_base.cc:1034] running on GCE node
I20260806 22:46:17.780757 8960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:17.781038 8960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:17.782182 8960 hybrid_clock.cc:648] HybridClock initialized: now 1786056377782172 us; error 29 us; skew 500 ppm
Aug 06 22:46:17 dist-test-slave-pgkr krb5kdc[8945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:17.784705 8960 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:17.785115 8960 webserver.cc:460] Webserver started at http://127.0.0.1:38977/ using document root <none> and password file <none>
I20260806 22:46:17.785323 8960 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:17.785367 8960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:17.785516 8960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:17.786510 8960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "2fec282a12c0492298a4b9550b591fda"
format_stamp: "Formatted at 2026-08-06 22:46:17 on dist-test-slave-pgkr"
server_key: "ccdadff15ddf97e339918171f92d6806"
server_key_iv: "ad1c16dd9e14c9181bafffedfca7460a"
server_key_version: "encryptionkey@0"
I20260806 22:46:17.786903 8960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "2fec282a12c0492298a4b9550b591fda"
format_stamp: "Formatted at 2026-08-06 22:46:17 on dist-test-slave-pgkr"
server_key: "ccdadff15ddf97e339918171f92d6806"
server_key_iv: "ad1c16dd9e14c9181bafffedfca7460a"
server_key_version: "encryptionkey@0"
I20260806 22:46:17.788166 8960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:17.788838 8977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:17.788992 8960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:17.789095 8960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "2fec282a12c0492298a4b9550b591fda"
format_stamp: "Formatted at 2026-08-06 22:46:17 on dist-test-slave-pgkr"
server_key: "ccdadff15ddf97e339918171f92d6806"
server_key_iv: "ad1c16dd9e14c9181bafffedfca7460a"
server_key_version: "encryptionkey@0"
I20260806 22:46:17.789163 8960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:17.800069 8960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:17.802232 8960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:17.802383 8960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:17.806272 8960 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38901
I20260806 22:46:17.806310 9035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38901 every 8 connection(s)
I20260806 22:46:17.806692 8960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:17.807512 9036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:17.809645 9036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda: Bootstrap starting.
I20260806 22:46:17.810166 9036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:17.810429 9036 log.cc:824] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:17.811141 9036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda: No bootstrap required, opened a new log
I20260806 22:46:17.812644 9036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fec282a12c0492298a4b9550b591fda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38901 } }
I20260806 22:46:17.812772 9036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:17.812795 9036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fec282a12c0492298a4b9550b591fda, State: Initialized, Role: FOLLOWER
I20260806 22:46:17.812927 9036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fec282a12c0492298a4b9550b591fda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38901 } }
I20260806 22:46:17.813010 9036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:17.813035 9036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:17.813067 9036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:17.813557 9036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fec282a12c0492298a4b9550b591fda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38901 } }
I20260806 22:46:17.813642 9036 leader_election.cc:302] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fec282a12c0492298a4b9550b591fda; no voters:
I20260806 22:46:17.813760 9036 leader_election.cc:288] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:17.813818 9039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:17.813980 9039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [term 1 LEADER]: Becoming Leader. State: Replica: 2fec282a12c0492298a4b9550b591fda, State: Running, Role: LEADER
I20260806 22:46:17.814000 9036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:17.814071 9039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fec282a12c0492298a4b9550b591fda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38901 } }
I20260806 22:46:17.814435 9041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fec282a12c0492298a4b9550b591fda. Latest consensus state: current_term: 1 leader_uuid: "2fec282a12c0492298a4b9550b591fda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fec282a12c0492298a4b9550b591fda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38901 } } }
I20260806 22:46:17.814539 9041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:17.814675 9040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fec282a12c0492298a4b9550b591fda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fec282a12c0492298a4b9550b591fda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38901 } } }
I20260806 22:46:17.814807 9040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:17.814891 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 8960
I20260806 22:46:17.815004 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:17.815224 19755 external_mini_cluster.cc:1428] Setting key e6f0f5db77f5bdc913bbab5bd307422c
I20260806 22:46:17.815274 9047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:17.815838 9047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:17.817505 9047 catalog_manager.cc:1347] Generated new cluster ID: 6280f189369f41329d4c980f6326e8d3
I20260806 22:46:17.817559 9047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:17.821442 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:17.815865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:42379 (local address 127.0.0.1:38901)
0806 22:46:17.816039 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:17.816048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:17.816080 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:46:17.816203 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:17.816210 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:17.816264 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:17.816403 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:17.816642 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:17.818174 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:17.819314 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:17.819937 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:17.820054 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:17.820745 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:17.820749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:17.820750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:17.820962 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:17.821000 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:17.821015 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:17.821108 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:17.821202 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:46:17.821231 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260806 22:46:17.822422 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:17.825029 9047 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:17.825702 9047 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:17.829731 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:17.826907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:56783 (local address 127.0.0.1:38901)
0806 22:46:17.826943 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:17.826945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:17.827071 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:46:17.827137 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:17.827138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:17.827150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:17.827196 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:17.827341 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:17.827845 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:17.828563 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:17.828807 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:17.828877 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:17.829491 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:17.829493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:17.829493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:17.829497 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:17.829511 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:17.829516 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:17.829568 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:17.829627 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:46:17.829638 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:46:17.834829 9047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fec282a12c0492298a4b9550b591fda: Generated new TSK 0
I20260806 22:46:17.834995 9047 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:17.840477 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 8960
W20260806 22:46:17.846318 8943 connection.cc:535] client connection to 127.0.0.1:38901 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38901): Connection reset by peer (error 104)
2026-08-06T22:46:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:17 dist-test-slave-pgkr krb5kdc[9068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:17 dist-test-slave-pgkr krb5kdc[9068](info): set up 2 sockets
Aug 06 22:46:17 dist-test-slave-pgkr krb5kdc[9068](info): commencing operation
krb5kdc: starting...
W20260806 22:46:18.881421 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:18 dist-test-slave-pgkr krb5kdc[9068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:19.000794 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40581
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:19.083442 9081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:19.083674 9081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:19.083711 9081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:19.084847 9081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:19.084908 9081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:19.084932 9081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:19.084959 9081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:19.084980 9081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:19.086922 9081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:19.087195 9081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:19.087428 9081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:19.092178 9086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:19.092244 9087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:19.092168 9089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:19.092391 9081 server_base.cc:1034] running on GCE node
I20260806 22:46:19.092620 9081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:19.092902 9081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:19.094071 9081 hybrid_clock.cc:648] HybridClock initialized: now 1786056379094058 us; error 28 us; skew 500 ppm
Aug 06 22:46:19 dist-test-slave-pgkr krb5kdc[9068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056379, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:19.096814 9081 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:19.097342 9081 webserver.cc:460] Webserver started at http://127.0.0.1:46429/ using document root <none> and password file <none>
I20260806 22:46:19.097574 9081 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:19.097620 9081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:19.097712 9081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:19.098840 9081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8"
format_stamp: "Formatted at 2026-08-06 22:46:19 on dist-test-slave-pgkr"
server_key: "18bfc31ad1cb31149b341cb5811f5065"
server_key_iv: "199f12a514f553780187edd0d3c5c3b4"
server_key_version: "encryptionkey@0"
I20260806 22:46:19.099179 9081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8"
format_stamp: "Formatted at 2026-08-06 22:46:19 on dist-test-slave-pgkr"
server_key: "18bfc31ad1cb31149b341cb5811f5065"
server_key_iv: "199f12a514f553780187edd0d3c5c3b4"
server_key_version: "encryptionkey@0"
I20260806 22:46:19.100592 9081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:19.101295 9096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:19.101472 9081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:19.101558 9081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8"
format_stamp: "Formatted at 2026-08-06 22:46:19 on dist-test-slave-pgkr"
server_key: "18bfc31ad1cb31149b341cb5811f5065"
server_key_iv: "199f12a514f553780187edd0d3c5c3b4"
server_key_version: "encryptionkey@0"
I20260806 22:46:19.101624 9081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:19.104820 9081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:19.106663 9081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:19.106810 9081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:19.110596 9081 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40581
I20260806 22:46:19.110707 9155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40581 every 8 connection(s)
I20260806 22:46:19.111017 9081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:19.111928 9156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:19.114130 9156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8: Bootstrap starting.
I20260806 22:46:19.114790 9156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:19.115085 9156 log.cc:824] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:19.115837 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 9081
I20260806 22:46:19.115873 9156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8: No bootstrap required, opened a new log
I20260806 22:46:19.115968 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:19.116174 19755 external_mini_cluster.cc:1428] Setting key 3295e930fbe11b3eb11e369fab357a4f
I20260806 22:46:19.117849 9156 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40581 } }
I20260806 22:46:19.118021 9156 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:19.118108 9156 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8cf7e1c3b5d47fe8d827750b83be4f8, State: Initialized, Role: FOLLOWER
I20260806 22:46:19.118253 9156 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40581 } }
I20260806 22:46:19.118364 9156 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:19.118422 9156 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:19.118477 9156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:19.119159 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:19.116689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:52703 (local address 127.0.0.1:40581)
0806 22:46:19.116872 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:19.116879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:19.116903 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:46:19.117827 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:19.117833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:19.117857 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:19.117969 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:19.118745 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:19.118748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:19.118749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:19.118753 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:19.118799 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:19.118810 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:19.118885 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:19.118955 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:46:19.118998 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260806 22:46:19.119560 9156 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40581 } }
I20260806 22:46:19.119683 9156 leader_election.cc:302] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8cf7e1c3b5d47fe8d827750b83be4f8; no voters:
I20260806 22:46:19.119846 9156 leader_election.cc:288] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:19.119895 9161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:19.120050 9161 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [term 1 LEADER]: Becoming Leader. State: Replica: b8cf7e1c3b5d47fe8d827750b83be4f8, State: Running, Role: LEADER
I20260806 22:46:19.120090 9156 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:19.120158 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:19.120229 9161 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40581 } }
I20260806 22:46:19.121526 9163 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8cf7e1c3b5d47fe8d827750b83be4f8. Latest consensus state: current_term: 1 leader_uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40581 } } }
I20260806 22:46:19.121832 9163 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:19.121970 9162 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cf7e1c3b5d47fe8d827750b83be4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40581 } } }
W20260806 22:46:19.122056 9181 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:19.122159 9181 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:19.122144 9162 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:19.123066 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:19.121764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:51163 (local address 127.0.0.1:40581)
0806 22:46:19.121792 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:19.121794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:19.122068 (+ 274us) server_negotiation.cc:407] Connection header received
0806 22:46:19.122202 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:19.122203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:19.122209 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:19.122254 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:19.122893 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:19.122895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:19.122895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:19.122899 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:19.122912 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:19.122916 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:19.122959 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:19.123016 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:46:19.123026 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260806 22:46:19.132273 9185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:19.132511 9185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:19.134155 9185 catalog_manager.cc:1347] Generated new cluster ID: 07a1ddfe4e3f43a8a44da69be26412e0
I20260806 22:46:19.134202 9185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:19.143169 9185 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:19.143353 9185 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:19.147320 9185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8cf7e1c3b5d47fe8d827750b83be4f8: Generated new TSK 0
I20260806 22:46:19.147495 9185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:19.153505 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 9081
2026-08-06T22:46:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:19 dist-test-slave-pgkr krb5kdc[9189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:19 dist-test-slave-pgkr krb5kdc[9189](info): set up 2 sockets
Aug 06 22:46:19 dist-test-slave-pgkr krb5kdc[9189](info): commencing operation
krb5kdc: starting...
W20260806 22:46:20.206676 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:20 dist-test-slave-pgkr krb5kdc[9189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:20.326015 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:20.407464 9202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:20.407708 9202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:20.407764 9202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:20.408906 9202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:20.408988 9202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:20.409013 9202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:20.409034 9202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:20.409062 9202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:20.410480 9202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40829
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:20.410784 9202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:20.411015 9202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:46:20.415799 9202 server_base.cc:1034] running on GCE node
W20260806 22:46:20.415767 9207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:20.415776 9210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:20.415757 9208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:20.416240 9202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:20.416509 9202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:20.417658 9202 hybrid_clock.cc:648] HybridClock initialized: now 1786056380417646 us; error 26 us; skew 500 ppm
Aug 06 22:46:20 dist-test-slave-pgkr krb5kdc[9189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:20.424911 9202 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:20.425330 9202 webserver.cc:460] Webserver started at http://127.0.0.1:38303/ using document root <none> and password file <none>
I20260806 22:46:20.425571 9202 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:20.425649 9202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:20.425794 9202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:20.426846 9202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "d212374975474a0a85b6a7d031f87e0d"
format_stamp: "Formatted at 2026-08-06 22:46:20 on dist-test-slave-pgkr"
server_key: "451d933aa02b0cd65f70d524b6a4136a"
server_key_iv: "3727837e9abde9608c6bfc4167d674c4"
server_key_version: "encryptionkey@0"
I20260806 22:46:20.427284 9202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "d212374975474a0a85b6a7d031f87e0d"
format_stamp: "Formatted at 2026-08-06 22:46:20 on dist-test-slave-pgkr"
server_key: "451d933aa02b0cd65f70d524b6a4136a"
server_key_iv: "3727837e9abde9608c6bfc4167d674c4"
server_key_version: "encryptionkey@0"
I20260806 22:46:20.428606 9202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:20.429325 9217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:20.429491 9202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:20.429607 9202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "d212374975474a0a85b6a7d031f87e0d"
format_stamp: "Formatted at 2026-08-06 22:46:20 on dist-test-slave-pgkr"
server_key: "451d933aa02b0cd65f70d524b6a4136a"
server_key_iv: "3727837e9abde9608c6bfc4167d674c4"
server_key_version: "encryptionkey@0"
I20260806 22:46:20.429682 9202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:20.439757 9202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:20.441479 9202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:20.441632 9202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:20.445632 9202 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40829
I20260806 22:46:20.445664 9271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40829 every 8 connection(s)
I20260806 22:46:20.445996 9202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:20.446885 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:20.449072 9272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d: Bootstrap starting.
I20260806 22:46:20.449645 9272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:20.449932 9272 log.cc:824] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:20.450603 9272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d: No bootstrap required, opened a new log
I20260806 22:46:20.450910 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 9202
I20260806 22:46:20.451047 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:20.451226 19755 external_mini_cluster.cc:1428] Setting key 6f37b9108a0126fc755aff0e9c8e3940
I20260806 22:46:20.452988 9272 raft_consensus.cc:348] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d212374975474a0a85b6a7d031f87e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } }
I20260806 22:46:20.453200 9272 raft_consensus.cc:374] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:20.453238 9272 raft_consensus.cc:729] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d212374975474a0a85b6a7d031f87e0d, State: Initialized, Role: FOLLOWER
I20260806 22:46:20.453361 9272 consensus_queue.cc:260] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d212374975474a0a85b6a7d031f87e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } }
I20260806 22:46:20.453483 9272 raft_consensus.cc:388] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:20.453518 9272 raft_consensus.cc:482] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:20.453567 9272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:20.454121 9272 raft_consensus.cc:504] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d212374975474a0a85b6a7d031f87e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } }
I20260806 22:46:20.454241 9272 leader_election.cc:302] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d212374975474a0a85b6a7d031f87e0d; no voters:
I20260806 22:46:20.454388 9272 leader_election.cc:288] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:20.454470 9277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:20.454676 9272 sys_catalog.cc:564] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:20.454684 9277 raft_consensus.cc:686] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [term 1 LEADER]: Becoming Leader. State: Replica: d212374975474a0a85b6a7d031f87e0d, State: Running, Role: LEADER
I20260806 22:46:20.454830 9277 consensus_queue.cc:237] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d212374975474a0a85b6a7d031f87e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } }
I20260806 22:46:20.455236 9278 sys_catalog.cc:455] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d212374975474a0a85b6a7d031f87e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d212374975474a0a85b6a7d031f87e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } } }
I20260806 22:46:20.455314 9278 sys_catalog.cc:458] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:20.455452 9278 sys_catalog.cc:455] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d212374975474a0a85b6a7d031f87e0d. Latest consensus state: current_term: 1 leader_uuid: "d212374975474a0a85b6a7d031f87e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d212374975474a0a85b6a7d031f87e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } } }
I20260806 22:46:20.455505 9278 sys_catalog.cc:458] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:20.455906 9287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:20.456478 9287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:20.458269 9287 catalog_manager.cc:1347] Generated new cluster ID: e30f69650ce446b9a89e3fdfbac21296
I20260806 22:46:20.458328 9287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:46:20 dist-test-slave-pgkr krb5kdc[9189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:46:20.463584 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:20.451765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:39565 (local address 127.0.0.1:40829)
0806 22:46:20.451989 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:20.451999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:20.452033 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:46:20.453190 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:20.453199 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:20.453264 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:20.453427 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:20.454079 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:20.455522 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:20.456655 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:20.457194 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:20.457317 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:20.460167 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:20.460172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:20.460172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:20.460183 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:20.461941 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:20.462360 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:20.462363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:20.462363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:20.462454 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:20.462720 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:20.462721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:20.462722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:20.463011 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:20.463171 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:20.463315 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 22:46:20.463406 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260806 22:46:20.464604 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:20.472836 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:20.465981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:50969 (local address 127.0.0.1:40829)
0806 22:46:20.466008 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:20.466010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:20.466135 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:46:20.466213 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:20.466214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:20.466222 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:20.466246 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:20.466852 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:20.467532 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:20.468474 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:20.468774 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:20.468807 (+ 33us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:20.470166 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:20.470167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:20.470168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:20.470174 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:20.471576 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:20.471977 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:20.471978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:20.471978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:20.472027 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:20.472265 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:20.472266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:20.472267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:20.472391 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:20.472459 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:20.472683 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 22:46:20.472749 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 22:46:20.473174 9287 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:20.473804 9287 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:20.482182 9287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d212374975474a0a85b6a7d031f87e0d: Generated new TSK 0
I20260806 22:46:20.482347 9287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:20.487653 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 9202
W20260806 22:46:20.493803 9187 connection.cc:535] client connection to 127.0.0.1:40829 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40829): Connection reset by peer (error 104)
2026-08-06T22:46:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:20 dist-test-slave-pgkr krb5kdc[9304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:20 dist-test-slave-pgkr krb5kdc[9304](info): set up 2 sockets
Aug 06 22:46:20 dist-test-slave-pgkr krb5kdc[9304](info): commencing operation
krb5kdc: starting...
W20260806 22:46:21.532944 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:21 dist-test-slave-pgkr krb5kdc[9304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:21.653019 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:21.735064 9317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:21.735302 9317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:21.735342 9317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:21.736469 9317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:21.736541 9317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:21.736563 9317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:21.736583 9317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:21.736624 9317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:21.738101 9317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:21.738372 9317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:21.738605 9317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:21.743404 9327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:21.743412 9324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:21.743412 9323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:21.743644 9317 server_base.cc:1034] running on GCE node
I20260806 22:46:21.743868 9317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:21.744145 9317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:21.745283 9317 hybrid_clock.cc:648] HybridClock initialized: now 1786056381745274 us; error 28 us; skew 500 ppm
Aug 06 22:46:21 dist-test-slave-pgkr krb5kdc[9304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:21.747779 9317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:21.748250 9317 webserver.cc:460] Webserver started at http://127.0.0.1:33269/ using document root <none> and password file <none>
I20260806 22:46:21.748458 9317 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:21.748498 9317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:21.748587 9317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:21.749542 9317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "1ea21f9a85094ccebda34a740b3ae0f6"
format_stamp: "Formatted at 2026-08-06 22:46:21 on dist-test-slave-pgkr"
server_key: "9d78ab828d7786dd7d58e0dfb0597aeb"
server_key_iv: "d96b8db2be44c08359d20a0665a137f8"
server_key_version: "encryptionkey@0"
I20260806 22:46:21.749907 9317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "1ea21f9a85094ccebda34a740b3ae0f6"
format_stamp: "Formatted at 2026-08-06 22:46:21 on dist-test-slave-pgkr"
server_key: "9d78ab828d7786dd7d58e0dfb0597aeb"
server_key_iv: "d96b8db2be44c08359d20a0665a137f8"
server_key_version: "encryptionkey@0"
I20260806 22:46:21.751242 9317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:21.751938 9334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:21.752107 9317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:21.752197 9317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "1ea21f9a85094ccebda34a740b3ae0f6"
format_stamp: "Formatted at 2026-08-06 22:46:21 on dist-test-slave-pgkr"
server_key: "9d78ab828d7786dd7d58e0dfb0597aeb"
server_key_iv: "d96b8db2be44c08359d20a0665a137f8"
server_key_version: "encryptionkey@0"
I20260806 22:46:21.752260 9317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:21.764073 9317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:21.765842 9317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:21.765949 9317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:21.769755 9317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36977
I20260806 22:46:21.769826 9386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36977 every 8 connection(s)
I20260806 22:46:21.770166 9317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:21.771168 9387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:21.773532 9387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6: Bootstrap starting.
I20260806 22:46:21.774161 9387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:21.774456 9387 log.cc:824] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:21.775447 9387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6: No bootstrap required, opened a new log
I20260806 22:46:21.777029 9387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea21f9a85094ccebda34a740b3ae0f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260806 22:46:21.777181 9387 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:21.777230 9387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea21f9a85094ccebda34a740b3ae0f6, State: Initialized, Role: FOLLOWER
I20260806 22:46:21.777370 9387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea21f9a85094ccebda34a740b3ae0f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260806 22:46:21.777449 9387 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:21.777493 9387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:21.777556 9387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:21.778124 9387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea21f9a85094ccebda34a740b3ae0f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260806 22:46:21.778246 9387 leader_election.cc:302] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ea21f9a85094ccebda34a740b3ae0f6; no voters:
I20260806 22:46:21.778292 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 9317
I20260806 22:46:21.778402 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:21.778434 9387 leader_election.cc:288] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:21.778486 9390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:21.778656 19755 external_mini_cluster.cc:1428] Setting key b75281a8a75dacf75772caf59a7350c1
I20260806 22:46:21.778667 9390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [term 1 LEADER]: Becoming Leader. State: Replica: 1ea21f9a85094ccebda34a740b3ae0f6, State: Running, Role: LEADER
I20260806 22:46:21.778723 9387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:21.778746 9390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea21f9a85094ccebda34a740b3ae0f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260806 22:46:21.779078 9392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ea21f9a85094ccebda34a740b3ae0f6. Latest consensus state: current_term: 1 leader_uuid: "1ea21f9a85094ccebda34a740b3ae0f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea21f9a85094ccebda34a740b3ae0f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } } }
I20260806 22:46:21.779202 9391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ea21f9a85094ccebda34a740b3ae0f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea21f9a85094ccebda34a740b3ae0f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } } }
I20260806 22:46:21.779229 9392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:21.779263 9391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6 [sys.catalog]: This master's current role is: LEADER
W20260806 22:46:21.780761 9407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:21.780817 9407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:21.780862 9399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:21.781024 9399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:21.782696 9399 catalog_manager.cc:1347] Generated new cluster ID: c435c391badd459584cf8982367bf82a
I20260806 22:46:21.782764 9399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:21.785275 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:21.779328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:40183 (local address 127.0.0.1:36977)
0806 22:46:21.780269 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:21.780291 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:21.780340 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:46:21.780470 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:21.780475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:21.780525 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:21.780662 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:21.780911 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:21.782076 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:21.783154 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:21.783748 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:21.783843 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:21.784610 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:21.784614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:21.784615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:21.784817 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:21.784855 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:21.784868 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:21.784944 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:21.785060 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:46:21.785084 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":62,"threads_started":1}
I20260806 22:46:21.786275 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:21.790928 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:21.787647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:39667 (local address 127.0.0.1:36977)
0806 22:46:21.787677 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:21.787679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:21.787797 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:46:21.787851 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:21.787851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:21.787869 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:21.787940 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:21.788076 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:21.788770 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:21.789713 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:21.789987 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:21.790039 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:21.790678 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:21.790680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:21.790680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:21.790684 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:21.790696 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:21.790699 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:21.790749 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:21.790838 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:46:21.790849 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:46:21.800105 9399 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:21.800607 9399 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:21.813818 9399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ea21f9a85094ccebda34a740b3ae0f6: Generated new TSK 0
I20260806 22:46:21.814080 9399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:21.817943 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 9317
W20260806 22:46:21.823985 9302 connection.cc:535] client connection to 127.0.0.1:36977 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36977): Connection reset by peer (error 104)
2026-08-06T22:46:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:21 dist-test-slave-pgkr krb5kdc[9419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:21 dist-test-slave-pgkr krb5kdc[9419](info): set up 2 sockets
Aug 06 22:46:21 dist-test-slave-pgkr krb5kdc[9419](info): commencing operation
krb5kdc: starting...
W20260806 22:46:22.864552 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:22 dist-test-slave-pgkr krb5kdc[9419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:46:22.990736 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:23.073181 9434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:23.073380 9434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:23.073402 9434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:23.074647 9434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:23.074705 9434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:23.074720 9434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:23.074779 9434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:23.074811 9434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:23.076272 9434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33053
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:23.076537 9434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:23.076794 9434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:23.081732 9440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:23.081739 9439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:23.081815 9434 server_base.cc:1034] running on GCE node
W20260806 22:46:23.081739 9442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:23.082178 9434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:23.082492 9434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:23.083685 9434 hybrid_clock.cc:648] HybridClock initialized: now 1786056383083663 us; error 48 us; skew 500 ppm
Aug 06 22:46:23 dist-test-slave-pgkr krb5kdc[9419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:23.086113 9434 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:46:23.086519 9434 webserver.cc:460] Webserver started at http://127.0.0.1:44977/ using document root <none> and password file <none>
I20260806 22:46:23.086771 9434 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:23.086849 9434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:23.086988 9434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:23.087954 9434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "3f4f89daa6f34edab3818f68065d5f38"
format_stamp: "Formatted at 2026-08-06 22:46:23 on dist-test-slave-pgkr"
server_key: "11f26e12b83a43bfdb5f7fb580bb82ff"
server_key_iv: "0998078a76a7f98c297995e81160481e"
server_key_version: "encryptionkey@0"
I20260806 22:46:23.088366 9434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "3f4f89daa6f34edab3818f68065d5f38"
format_stamp: "Formatted at 2026-08-06 22:46:23 on dist-test-slave-pgkr"
server_key: "11f26e12b83a43bfdb5f7fb580bb82ff"
server_key_iv: "0998078a76a7f98c297995e81160481e"
server_key_version: "encryptionkey@0"
I20260806 22:46:23.089691 9434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:23.090405 9449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:23.090579 9434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:23.090701 9434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "3f4f89daa6f34edab3818f68065d5f38"
format_stamp: "Formatted at 2026-08-06 22:46:23 on dist-test-slave-pgkr"
server_key: "11f26e12b83a43bfdb5f7fb580bb82ff"
server_key_iv: "0998078a76a7f98c297995e81160481e"
server_key_version: "encryptionkey@0"
I20260806 22:46:23.090785 9434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:23.093992 9434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:23.095815 9434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:23.095968 9434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:23.100144 9434 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33053
I20260806 22:46:23.100183 9502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33053 every 8 connection(s)
I20260806 22:46:23.100483 9434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:23.101323 9504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:23.103593 9504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38: Bootstrap starting.
I20260806 22:46:23.104172 9504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:23.104453 9504 log.cc:824] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:23.105105 9504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38: No bootstrap required, opened a new log
I20260806 22:46:23.105540 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 9434
I20260806 22:46:23.105664 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:23.105862 19755 external_mini_cluster.cc:1428] Setting key 3bd8443892106995f175559faa91a8d5
I20260806 22:46:23.107455 9504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4f89daa6f34edab3818f68065d5f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33053 } }
I20260806 22:46:23.107582 9504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:23.107604 9504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f4f89daa6f34edab3818f68065d5f38, State: Initialized, Role: FOLLOWER
I20260806 22:46:23.107728 9504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4f89daa6f34edab3818f68065d5f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33053 } }
I20260806 22:46:23.107821 9504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:23.107854 9504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:23.107959 9504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:23.108963 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:23.106320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:34273 (local address 127.0.0.1:33053)
0806 22:46:23.106488 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:23.106495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:23.106519 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:46:23.107628 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:23.107639 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:23.107681 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:23.107842 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:23.108528 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:23.108530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:23.108531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:23.108534 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:23.108565 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:23.108574 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:23.108633 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:23.108725 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:46:23.108775 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260806 22:46:23.109020 9504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4f89daa6f34edab3818f68065d5f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33053 } }
I20260806 22:46:23.109172 9504 leader_election.cc:302] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f4f89daa6f34edab3818f68065d5f38; no voters:
I20260806 22:46:23.109339 9504 leader_election.cc:288] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:23.109378 9509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:23.109539 9509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [term 1 LEADER]: Becoming Leader. State: Replica: 3f4f89daa6f34edab3818f68065d5f38, State: Running, Role: LEADER
I20260806 22:46:23.109617 9504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:23.109690 9509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4f89daa6f34edab3818f68065d5f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33053 } }
I20260806 22:46:23.109791 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:23.110057 9509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f4f89daa6f34edab3818f68065d5f38. Latest consensus state: current_term: 1 leader_uuid: "3f4f89daa6f34edab3818f68065d5f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4f89daa6f34edab3818f68065d5f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33053 } } }
I20260806 22:46:23.110067 9510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f4f89daa6f34edab3818f68065d5f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4f89daa6f34edab3818f68065d5f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33053 } } }
I20260806 22:46:23.110149 9509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:23.110172 9510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:23.110395 9517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:23.110925 9517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:23.112664 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:23.111372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.175:41269 (local address 127.0.0.1:33053)
0806 22:46:23.111406 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:23.111408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:23.111644 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:46:23.111700 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:23.111700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:23.111705 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:23.111952 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:23.112373 (+ 421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:23.112374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:23.112374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:23.112379 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:23.112392 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:23.112397 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:23.112578 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:23.112594 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 22:46:23.112606 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:23.112746 9517 catalog_manager.cc:1347] Generated new cluster ID: 2008e5acc38a4f988debddfe208d6d6d
I20260806 22:46:23.112852 9517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:23.132700 9517 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:23.132884 9517 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:23.138062 9517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f4f89daa6f34edab3818f68065d5f38: Generated new TSK 0
I20260806 22:46:23.138257 9517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:23.148983 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 9434
2026-08-06T22:46:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:23 dist-test-slave-pgkr krb5kdc[9536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:23 dist-test-slave-pgkr krb5kdc[9536](info): set up 2 sockets
Aug 06 22:46:23 dist-test-slave-pgkr krb5kdc[9536](info): commencing operation
krb5kdc: starting...
W20260806 22:46:24.224958 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:24 dist-test-slave-pgkr krb5kdc[9536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:24.343127 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:45289
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:45289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:24.424891 9549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:24.425148 9549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:24.425184 9549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:24.426366 9549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:24.426430 9549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:24.426455 9549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:24.426484 9549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:24.426527 9549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:24.428009 9549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:45289
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:45289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:24.428293 9549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:24.428520 9549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:24.433387 9555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:24.433437 9557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:24.433409 9554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:24.433620 9549 server_base.cc:1034] running on GCE node
I20260806 22:46:24.433929 9549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:24.434226 9549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:24.435387 9549 hybrid_clock.cc:648] HybridClock initialized: now 1786056384435375 us; error 32 us; skew 500 ppm
Aug 06 22:46:24 dist-test-slave-pgkr krb5kdc[9536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056384, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:24.437813 9549 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:24.438285 9549 webserver.cc:460] Webserver started at http://127.19.74.254:38075/ using document root <none> and password file <none>
I20260806 22:46:24.438474 9549 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:24.438514 9549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:24.438655 9549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:24.439684 9549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "37c2c0f537cf4fdfa9098e9168d82053"
format_stamp: "Formatted at 2026-08-06 22:46:24 on dist-test-slave-pgkr"
server_key: "6b54376cc4a53a6cc37a47c866512828"
server_key_iv: "5be63b9c285047989ef5dcce76efed2a"
server_key_version: "encryptionkey@0"
I20260806 22:46:24.439980 9549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "37c2c0f537cf4fdfa9098e9168d82053"
format_stamp: "Formatted at 2026-08-06 22:46:24 on dist-test-slave-pgkr"
server_key: "6b54376cc4a53a6cc37a47c866512828"
server_key_iv: "5be63b9c285047989ef5dcce76efed2a"
server_key_version: "encryptionkey@0"
I20260806 22:46:24.441258 9549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:24.441952 9566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:24.442121 9549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:24.442227 9549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "37c2c0f537cf4fdfa9098e9168d82053"
format_stamp: "Formatted at 2026-08-06 22:46:24 on dist-test-slave-pgkr"
server_key: "6b54376cc4a53a6cc37a47c866512828"
server_key_iv: "5be63b9c285047989ef5dcce76efed2a"
server_key_version: "encryptionkey@0"
I20260806 22:46:24.442298 9549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:24.476547 9549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:24.479104 9549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:24.479283 9549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:24.483480 9549 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:45289
I20260806 22:46:24.483520 9623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:45289 every 8 connection(s)
I20260806 22:46:24.483909 9549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:24.484778 9624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:24.487067 9624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053: Bootstrap starting.
I20260806 22:46:24.487622 9624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:24.487857 9624 log.cc:824] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:24.488292 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 9549
I20260806 22:46:24.488416 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:24.488616 19755 external_mini_cluster.cc:1428] Setting key 417e1d46ee8f1046e9506de24c7b0202
I20260806 22:46:24.488634 9624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053: No bootstrap required, opened a new log
I20260806 22:46:24.491036 9624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c2c0f537cf4fdfa9098e9168d82053" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45289 } }
I20260806 22:46:24.491179 9624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:24.491199 9624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c2c0f537cf4fdfa9098e9168d82053, State: Initialized, Role: FOLLOWER
I20260806 22:46:24.491318 9624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c2c0f537cf4fdfa9098e9168d82053" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45289 } }
I20260806 22:46:24.491387 9624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:24.491408 9624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:24.491464 9624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:24.491971 9624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c2c0f537cf4fdfa9098e9168d82053" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45289 } }
I20260806 22:46:24.492054 9624 leader_election.cc:302] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37c2c0f537cf4fdfa9098e9168d82053; no voters:
I20260806 22:46:24.492246 9624 leader_election.cc:288] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:24.492268 9629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:24.492467 9629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [term 1 LEADER]: Becoming Leader. State: Replica: 37c2c0f537cf4fdfa9098e9168d82053, State: Running, Role: LEADER
I20260806 22:46:24.492552 9629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c2c0f537cf4fdfa9098e9168d82053" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45289 } }
I20260806 22:46:24.492583 9624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:24.492914 9629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37c2c0f537cf4fdfa9098e9168d82053. Latest consensus state: current_term: 1 leader_uuid: "37c2c0f537cf4fdfa9098e9168d82053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c2c0f537cf4fdfa9098e9168d82053" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45289 } } }
I20260806 22:46:24.492986 9629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:24.492923 9631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37c2c0f537cf4fdfa9098e9168d82053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c2c0f537cf4fdfa9098e9168d82053" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 45289 } } }
I20260806 22:46:24.493038 9631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:24.493383 9634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:24.493973 9634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:46:24 dist-test-slave-pgkr krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:24.496747 9634 catalog_manager.cc:1347] Generated new cluster ID: be6a6cfaee204b11986bfe1b6fce3c07
I20260806 22:46:24.496794 9634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:24.500561 9628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:24.489111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44031 (local address 127.19.74.254:45289)
0806 22:46:24.489302 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:24.489309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:24.489334 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:46:24.489955 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:24.489963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:24.490012 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:24.490120 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:24.490929 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:24.492200 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:24.493286 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:24.493917 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:24.493996 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:24.496855 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:24.496859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:24.496860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:24.496871 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:24.498871 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:24.499321 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:24.499324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:24.499324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:24.499441 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:24.499676 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:24.499677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:24.499677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:24.499976 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:24.500141 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:24.500256 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 22:46:24.500361 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260806 22:46:24.501653 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:24.509302 9634 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:24.509768 9634 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:24.516098 9628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:24.503061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55571 (local address 127.19.74.254:45289)
0806 22:46:24.503126 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:24.503128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:24.509107 (+ 5979us) server_negotiation.cc:407] Connection header received
0806 22:46:24.509185 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:24.509186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:24.509196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:24.509266 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:24.509913 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:24.510599 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:24.511604 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:24.511884 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:24.511942 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:24.513351 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:24.513352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:24.513352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:24.513359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:24.514879 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:24.515294 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:24.515294 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:24.515295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:24.515348 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:24.515584 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:24.515584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:24.515585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:24.515699 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:24.515793 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:24.515962 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:46:24.516025 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":45}
I20260806 22:46:24.516594 9634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37c2c0f537cf4fdfa9098e9168d82053: Generated new TSK 0
I20260806 22:46:24.516755 9634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:24.520592 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 9549
W20260806 22:46:24.527341 9534 connection.cc:535] client connection to 127.19.74.254:45289 recv error: Network error: failed to read from TLS socket (remote: 127.19.74.254:45289): Connection reset by peer (error 104)
2026-08-06T22:46:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:24 dist-test-slave-pgkr krb5kdc[9656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:24 dist-test-slave-pgkr krb5kdc[9656](info): set up 2 sockets
Aug 06 22:46:24 dist-test-slave-pgkr krb5kdc[9656](info): commencing operation
krb5kdc: starting...
W20260806 22:46:25.557065 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:25 dist-test-slave-pgkr krb5kdc[9656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:25.676672 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:33527
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:33527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:25.759990 9671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:25.760201 9671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:25.760226 9671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:25.761551 9671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:25.761606 9671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:25.761623 9671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:25.761638 9671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:25.761652 9671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:25.763180 9671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:33527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:33527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:25.763460 9671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:25.763720 9671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:25.768926 9679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:25.768937 9677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:25.769107 9671 server_base.cc:1034] running on GCE node
W20260806 22:46:25.768942 9676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:25.769434 9671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:25.769781 9671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:25.770933 9671 hybrid_clock.cc:648] HybridClock initialized: now 1786056385770914 us; error 39 us; skew 500 ppm
Aug 06 22:46:25 dist-test-slave-pgkr krb5kdc[9656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056385, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:25.779385 9671 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:25.779896 9671 webserver.cc:460] Webserver started at http://127.19.74.254:36945/ using document root <none> and password file <none>
I20260806 22:46:25.780210 9671 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:25.780257 9671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:25.780404 9671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:25.781522 9671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "d5497094dd70457eb4b2d6db7dc21440"
format_stamp: "Formatted at 2026-08-06 22:46:25 on dist-test-slave-pgkr"
server_key: "82f3591fc833622193aea77ee66de156"
server_key_iv: "15b0029df75d418a66b95c362f7936c4"
server_key_version: "encryptionkey@0"
I20260806 22:46:25.781893 9671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "d5497094dd70457eb4b2d6db7dc21440"
format_stamp: "Formatted at 2026-08-06 22:46:25 on dist-test-slave-pgkr"
server_key: "82f3591fc833622193aea77ee66de156"
server_key_iv: "15b0029df75d418a66b95c362f7936c4"
server_key_version: "encryptionkey@0"
I20260806 22:46:25.783351 9671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:46:25.784185 9686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:25.784405 9671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:25.784482 9671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "d5497094dd70457eb4b2d6db7dc21440"
format_stamp: "Formatted at 2026-08-06 22:46:25 on dist-test-slave-pgkr"
server_key: "82f3591fc833622193aea77ee66de156"
server_key_iv: "15b0029df75d418a66b95c362f7936c4"
server_key_version: "encryptionkey@0"
I20260806 22:46:25.784570 9671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:25.808208 9671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:25.810753 9671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:25.810916 9671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:25.815174 9671 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:33527
I20260806 22:46:25.815217 9741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:33527 every 8 connection(s)
I20260806 22:46:25.815626 9671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:25.816515 9742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:25.818836 9742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440: Bootstrap starting.
I20260806 22:46:25.819437 9742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:25.819732 9742 log.cc:824] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:25.820416 9742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440: No bootstrap required, opened a new log
I20260806 22:46:25.822160 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 9671
I20260806 22:46:25.822054 9742 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5497094dd70457eb4b2d6db7dc21440" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33527 } }
I20260806 22:46:25.822229 9742 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:25.822271 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:25.822268 9742 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5497094dd70457eb4b2d6db7dc21440, State: Initialized, Role: FOLLOWER
I20260806 22:46:25.822423 9742 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5497094dd70457eb4b2d6db7dc21440" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33527 } }
I20260806 22:46:25.822518 19755 external_mini_cluster.cc:1428] Setting key a8d97335e219480bb9848d54cc47cb7c
I20260806 22:46:25.822539 9742 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:25.822589 9742 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:25.822675 9742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:25.823782 9742 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5497094dd70457eb4b2d6db7dc21440" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33527 } }
I20260806 22:46:25.823894 9742 leader_election.cc:302] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5497094dd70457eb4b2d6db7dc21440; no voters:
I20260806 22:46:25.824083 9742 leader_election.cc:288] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:25.824115 9747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:25.824301 9747 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [term 1 LEADER]: Becoming Leader. State: Replica: d5497094dd70457eb4b2d6db7dc21440, State: Running, Role: LEADER
I20260806 22:46:25.824390 9742 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:25.824445 9747 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5497094dd70457eb4b2d6db7dc21440" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33527 } }
I20260806 22:46:25.824790 9747 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5497094dd70457eb4b2d6db7dc21440. Latest consensus state: current_term: 1 leader_uuid: "d5497094dd70457eb4b2d6db7dc21440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5497094dd70457eb4b2d6db7dc21440" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33527 } } }
I20260806 22:46:25.824803 9748 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5497094dd70457eb4b2d6db7dc21440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5497094dd70457eb4b2d6db7dc21440" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33527 } } }
I20260806 22:46:25.824931 9747 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:25.825029 9748 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:25.825296 9757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:25.825881 9757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:25.827667 9757 catalog_manager.cc:1347] Generated new cluster ID: f537c3426b9e46d6adde86e5a2b2cfe7
I20260806 22:46:25.827772 9757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:25.828510 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:25.823026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49371 (local address 127.19.74.254:33527)
0806 22:46:25.823222 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:25.823232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:25.823262 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:46:25.823360 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:25.823365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:25.823404 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:25.823509 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:25.823745 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:25.825150 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:25.826307 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:25.826947 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:25.827059 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:25.827800 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:25.827804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:25.827804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:25.828009 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:25.828047 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:25.828062 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:25.828169 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:25.828273 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:46:25.828301 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260806 22:46:25.829492 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:25.834508 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:25.830996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59143 (local address 127.19.74.254:33527)
0806 22:46:25.831038 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:25.831041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:25.831188 (+ 147us) server_negotiation.cc:407] Connection header received
0806 22:46:25.831247 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:25.831248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:25.831257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:25.831305 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:25.831430 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:25.832204 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:25.833237 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:25.833534 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:25.833586 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:25.834263 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:25.834264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:25.834265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:25.834269 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:25.834282 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:25.834285 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:25.834336 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:25.834433 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:46:25.834444 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:46:25.859508 9757 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:25.860219 9757 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:25.862994 9757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5497094dd70457eb4b2d6db7dc21440: Generated new TSK 0
I20260806 22:46:25.863174 9757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:25.867702 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 9671
W20260806 22:46:25.873971 9654 connection.cc:535] client connection to 127.19.74.254:33527 recv error: Network error: failed to read from TLS socket (remote: 127.19.74.254:33527): Connection reset by peer (error 104)
2026-08-06T22:46:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1346 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:25 dist-test-slave-pgkr krb5kdc[9774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:25 dist-test-slave-pgkr krb5kdc[9774](info): set up 2 sockets
Aug 06 22:46:25 dist-test-slave-pgkr krb5kdc[9774](info): commencing operation
krb5kdc: starting...
W20260806 22:46:26.913527 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:26 dist-test-slave-pgkr krb5kdc[9774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:27.031193 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42763
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:35889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:42763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:27.113407 9789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:27.113648 9789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:27.113689 9789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:27.114877 9789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:27.114950 9789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:27.114979 9789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:27.115023 9789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:27.115049 9789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:27.116531 9789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:35889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:42763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:27.116802 9789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:27.117050 9789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:27.122146 9797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:27.122145 9795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:27.122501 9794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:27.122666 9789 server_base.cc:1034] running on GCE node
I20260806 22:46:27.122922 9789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:27.123208 9789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:27.124385 9789 hybrid_clock.cc:648] HybridClock initialized: now 1786056387124361 us; error 38 us; skew 500 ppm
Aug 06 22:46:27 dist-test-slave-pgkr krb5kdc[9774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056387, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:27.126924 9789 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:27.127279 9789 webserver.cc:460] Webserver started at http://127.19.74.254:33521/ using document root <none> and password file <none>
I20260806 22:46:27.127439 9789 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:27.127475 9789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:27.127554 9789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:27.128504 9789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "aba079d361ad49a4abcf5b7f0ece693c"
format_stamp: "Formatted at 2026-08-06 22:46:27 on dist-test-slave-pgkr"
server_key: "e6a2f5c0a046fefde3c287524aa560f6"
server_key_iv: "42e0362cfcfaa36d459f51d7dd1f778c"
server_key_version: "encryptionkey@0"
I20260806 22:46:27.128811 9789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "aba079d361ad49a4abcf5b7f0ece693c"
format_stamp: "Formatted at 2026-08-06 22:46:27 on dist-test-slave-pgkr"
server_key: "e6a2f5c0a046fefde3c287524aa560f6"
server_key_iv: "42e0362cfcfaa36d459f51d7dd1f778c"
server_key_version: "encryptionkey@0"
I20260806 22:46:27.130072 9789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:27.130776 9805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:27.130932 9789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:27.131022 9789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "aba079d361ad49a4abcf5b7f0ece693c"
format_stamp: "Formatted at 2026-08-06 22:46:27 on dist-test-slave-pgkr"
server_key: "e6a2f5c0a046fefde3c287524aa560f6"
server_key_iv: "42e0362cfcfaa36d459f51d7dd1f778c"
server_key_version: "encryptionkey@0"
I20260806 22:46:27.131076 9789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:27.134238 9789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:27.136358 9789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:27.136461 9789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:27.140313 9789 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:42763
I20260806 22:46:27.140379 9859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:42763 every 8 connection(s)
I20260806 22:46:27.140661 9789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:27.141661 9860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:27.143970 9860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c: Bootstrap starting.
I20260806 22:46:27.144550 9860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:27.144822 9860 log.cc:824] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:27.145478 9860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c: No bootstrap required, opened a new log
I20260806 22:46:27.145648 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 9789
I20260806 22:46:27.145750 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:27.145936 19755 external_mini_cluster.cc:1428] Setting key cc88dfea8a6cd4d7c9e8ad78608f4adc
I20260806 22:46:27.148169 9860 raft_consensus.cc:348] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba079d361ad49a4abcf5b7f0ece693c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42763 } }
I20260806 22:46:27.148343 9860 raft_consensus.cc:374] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:27.148401 9860 raft_consensus.cc:729] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba079d361ad49a4abcf5b7f0ece693c, State: Initialized, Role: FOLLOWER
I20260806 22:46:27.148583 9860 consensus_queue.cc:260] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba079d361ad49a4abcf5b7f0ece693c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42763 } }
I20260806 22:46:27.148707 9860 raft_consensus.cc:388] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:27.148769 9860 raft_consensus.cc:482] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:27.148823 9860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:27.149379 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:27.146443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46337 (local address 127.19.74.254:42763)
0806 22:46:27.146681 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:27.146691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:27.146724 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:46:27.148117 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:27.148127 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:27.148163 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:27.148319 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:27.148960 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:27.148963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:27.148964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:27.148968 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:27.149008 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:27.149029 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:27.149102 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:27.149181 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:46:27.149224 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":108,"threads_started":1}
I20260806 22:46:27.149408 9860 raft_consensus.cc:504] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba079d361ad49a4abcf5b7f0ece693c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42763 } }
I20260806 22:46:27.149538 9860 leader_election.cc:302] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aba079d361ad49a4abcf5b7f0ece693c; no voters:
I20260806 22:46:27.149700 9860 leader_election.cc:288] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:27.149753 9865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:27.149907 9865 raft_consensus.cc:686] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [term 1 LEADER]: Becoming Leader. State: Replica: aba079d361ad49a4abcf5b7f0ece693c, State: Running, Role: LEADER
I20260806 22:46:27.149948 9860 sys_catalog.cc:564] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:27.150058 9865 consensus_queue.cc:237] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba079d361ad49a4abcf5b7f0ece693c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42763 } }
I20260806 22:46:27.150290 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:27.150404 9866 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aba079d361ad49a4abcf5b7f0ece693c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba079d361ad49a4abcf5b7f0ece693c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42763 } } }
I20260806 22:46:27.150472 9866 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:27.150629 9866 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [sys.catalog]: SysCatalogTable state changed. Reason: New leader aba079d361ad49a4abcf5b7f0ece693c. Latest consensus state: current_term: 1 leader_uuid: "aba079d361ad49a4abcf5b7f0ece693c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba079d361ad49a4abcf5b7f0ece693c" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42763 } } }
I20260806 22:46:27.150672 9866 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:27.151248 9872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:27.151738 9872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:27.153566 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:27.151999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38541 (local address 127.19.74.254:42763)
0806 22:46:27.152031 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:27.152033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:27.152649 (+ 616us) server_negotiation.cc:407] Connection header received
0806 22:46:27.152685 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:27.152686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:27.152702 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:27.152762 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:27.153295 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:27.153297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:27.153297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:27.153313 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:27.153328 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:27.153332 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:27.153411 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:27.153476 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:46:27.153491 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:46:27.153767 9872 catalog_manager.cc:1347] Generated new cluster ID: 839b00aa66054f56a085c4435eb0724e
I20260806 22:46:27.153824 9872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:27.186475 9872 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:27.186712 9872 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:27.189877 9872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aba079d361ad49a4abcf5b7f0ece693c: Generated new TSK 0
I20260806 22:46:27.190059 9872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:27.196099 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 9789
2026-08-06T22:46:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:27 dist-test-slave-pgkr krb5kdc[9895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:27 dist-test-slave-pgkr krb5kdc[9895](info): set up 2 sockets
Aug 06 22:46:27 dist-test-slave-pgkr krb5kdc[9895](info): commencing operation
krb5kdc: starting...
W20260806 22:46:28.248975 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:28 dist-test-slave-pgkr krb5kdc[9895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:28.367291 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:46247
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:40213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:46247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:28.448585 9908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:28.448781 9908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:28.448803 9908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:28.449915 9908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:28.449955 9908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:28.449968 9908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:28.450007 9908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:28.450035 9908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:28.451427 9908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:40213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:46247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:46247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:28.451680 9908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:28.451906 9908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:28.456797 9915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:28.456812 9917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:28.456838 9914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:28.457187 9908 server_base.cc:1034] running on GCE node
I20260806 22:46:28.457422 9908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:28.457711 9908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:28.458875 9908 hybrid_clock.cc:648] HybridClock initialized: now 1786056388458869 us; error 36 us; skew 500 ppm
Aug 06 22:46:28 dist-test-slave-pgkr krb5kdc[9895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056388, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:28.461341 9908 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:28.461817 9908 webserver.cc:460] Webserver started at http://127.19.74.254:43365/ using document root <none> and password file <none>
I20260806 22:46:28.462064 9908 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:28.462129 9908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:28.462283 9908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:28.463333 9908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "4f5c8b14446b4835bd83200976bba64a"
format_stamp: "Formatted at 2026-08-06 22:46:28 on dist-test-slave-pgkr"
server_key: "56fb07b7c05cda43a599c938eb40c0ab"
server_key_iv: "44686eef79c6e6ff732a33cd2efd7fa6"
server_key_version: "encryptionkey@0"
I20260806 22:46:28.463698 9908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "4f5c8b14446b4835bd83200976bba64a"
format_stamp: "Formatted at 2026-08-06 22:46:28 on dist-test-slave-pgkr"
server_key: "56fb07b7c05cda43a599c938eb40c0ab"
server_key_iv: "44686eef79c6e6ff732a33cd2efd7fa6"
server_key_version: "encryptionkey@0"
I20260806 22:46:28.464977 9908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:46:28.465685 9924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:28.465855 9908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:28.465957 9908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "4f5c8b14446b4835bd83200976bba64a"
format_stamp: "Formatted at 2026-08-06 22:46:28 on dist-test-slave-pgkr"
server_key: "56fb07b7c05cda43a599c938eb40c0ab"
server_key_iv: "44686eef79c6e6ff732a33cd2efd7fa6"
server_key_version: "encryptionkey@0"
I20260806 22:46:28.466035 9908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:28.472647 9908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:28.474896 9908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:28.475049 9908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:28.478883 9908 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:46247
I20260806 22:46:28.478958 9978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:46247 every 8 connection(s)
I20260806 22:46:28.479280 9908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:28.480150 9979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:28.481753 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 9908
I20260806 22:46:28.481864 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:28.482071 19755 external_mini_cluster.cc:1428] Setting key 7cd12d9dea76f0698fb3e312c16aea81
I20260806 22:46:28.482815 9979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a: Bootstrap starting.
I20260806 22:46:28.483443 9979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:28.483758 9979 log.cc:824] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:28.484491 9979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a: No bootstrap required, opened a new log
I20260806 22:46:28.486765 9979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f5c8b14446b4835bd83200976bba64a" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46247 } }
I20260806 22:46:28.486944 9979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:28.487007 9979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f5c8b14446b4835bd83200976bba64a, State: Initialized, Role: FOLLOWER
I20260806 22:46:28.487106 9979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f5c8b14446b4835bd83200976bba64a" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46247 } }
I20260806 22:46:28.487221 9979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:28.487258 9979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:28.487315 9979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:28.488080 9979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f5c8b14446b4835bd83200976bba64a" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46247 } }
I20260806 22:46:28.488201 9979 leader_election.cc:302] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f5c8b14446b4835bd83200976bba64a; no voters:
I20260806 22:46:28.488374 9979 leader_election.cc:288] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:28.488422 9985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:28.488601 9985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [term 1 LEADER]: Becoming Leader. State: Replica: 4f5c8b14446b4835bd83200976bba64a, State: Running, Role: LEADER
I20260806 22:46:28.488700 9979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:28.488751 9985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f5c8b14446b4835bd83200976bba64a" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46247 } }
Aug 06 22:46:28 dist-test-slave-pgkr krb5kdc[9895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:28.489238 9986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f5c8b14446b4835bd83200976bba64a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f5c8b14446b4835bd83200976bba64a" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46247 } } }
I20260806 22:46:28.489326 9986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:28.489569 9987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f5c8b14446b4835bd83200976bba64a. Latest consensus state: current_term: 1 leader_uuid: "4f5c8b14446b4835bd83200976bba64a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f5c8b14446b4835bd83200976bba64a" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 46247 } } }
I20260806 22:46:28.489714 9987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a [sys.catalog]: This master's current role is: LEADER
W20260806 22:46:28.490582 10001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:28.490676 10001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:28.490721 9994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:28.490873 9994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:28.492552 9994 catalog_manager.cc:1347] Generated new cluster ID: e2e65e0db5d34900bf1829bbc07e203b
I20260806 22:46:28.492607 9994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:28.493682 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:28.482557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42791 (local address 127.19.74.254:46247)
0806 22:46:28.482748 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:28.482756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:28.482780 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:46:28.483243 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:28.483250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:28.483295 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:28.483404 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:28.484190 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:28.485094 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:28.486144 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:28.486529 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:28.486670 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:28.489652 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:28.489656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:28.489657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:28.489670 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:28.491619 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:28.492036 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:28.492039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:28.492040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:28.492146 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:28.492395 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:28.492396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:28.492396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:28.492724 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:28.493271 (+ 547us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:28.493344 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 22:46:28.493465 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260806 22:46:28.494663 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:28.504041 9994 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:28.504701 9994 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:28.506068 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:28.498942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38953 (local address 127.19.74.254:46247)
0806 22:46:28.498979 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:28.498981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:28.499126 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:46:28.499185 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:28.499186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:28.499197 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:28.499242 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:28.499907 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:28.500654 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:28.501674 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:28.501989 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:28.502048 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:28.503426 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:28.503428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:28.503428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:28.503435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:28.504988 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:28.505328 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:28.505329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:28.505329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:28.505386 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:28.505582 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:28.505583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:28.505583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:28.505686 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:28.505783 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:28.505891 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:46:28.505969 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:46:28.514047 9994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f5c8b14446b4835bd83200976bba64a: Generated new TSK 0
I20260806 22:46:28.514247 9994 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:28.517323 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 9908
2026-08-06T22:46:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:28 dist-test-slave-pgkr krb5kdc[10011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:28 dist-test-slave-pgkr krb5kdc[10011](info): set up 2 sockets
Aug 06 22:46:28 dist-test-slave-pgkr krb5kdc[10011](info): commencing operation
krb5kdc: starting...
W20260806 22:46:29.572458 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:29 dist-test-slave-pgkr krb5kdc[10011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:29.690455 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42465
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:42465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:29.771239 10025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:29.771423 10025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:29.771445 10025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:29.772609 10025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:29.772652 10025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:29.772666 10025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:29.772707 10025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:29.772738 10025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:29.774250 10025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:42465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:29.774501 10025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:29.774879 10025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:29.779814 10033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:29.779897 10030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:29.779907 10025 server_base.cc:1034] running on GCE node
W20260806 22:46:29.779850 10031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:29.780210 10025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:29.780498 10025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:29.781692 10025 hybrid_clock.cc:648] HybridClock initialized: now 1786056389781670 us; error 33 us; skew 500 ppm
Aug 06 22:46:29 dist-test-slave-pgkr krb5kdc[10011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056389, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:29.784124 10025 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:29.784616 10025 webserver.cc:460] Webserver started at http://127.19.74.254:37791/ using document root <none> and password file <none>
I20260806 22:46:29.784834 10025 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:29.784901 10025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:29.785053 10025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:29.786062 10025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "53f9b2a51fb44df3a5d9af00f6cb916b"
format_stamp: "Formatted at 2026-08-06 22:46:29 on dist-test-slave-pgkr"
server_key: "05588a90fbdae3913884cef502ab2a41"
server_key_iv: "8603c4033a1587395ce3dafcbc502607"
server_key_version: "encryptionkey@0"
I20260806 22:46:29.786422 10025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "53f9b2a51fb44df3a5d9af00f6cb916b"
format_stamp: "Formatted at 2026-08-06 22:46:29 on dist-test-slave-pgkr"
server_key: "05588a90fbdae3913884cef502ab2a41"
server_key_iv: "8603c4033a1587395ce3dafcbc502607"
server_key_version: "encryptionkey@0"
I20260806 22:46:29.787832 10025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:29.788655 10041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:29.788846 10025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:29.788947 10025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "53f9b2a51fb44df3a5d9af00f6cb916b"
format_stamp: "Formatted at 2026-08-06 22:46:29 on dist-test-slave-pgkr"
server_key: "05588a90fbdae3913884cef502ab2a41"
server_key_iv: "8603c4033a1587395ce3dafcbc502607"
server_key_version: "encryptionkey@0"
I20260806 22:46:29.789039 10025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:29.802184 10025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:29.804005 10025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:29.804158 10025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:29.808372 10025 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:42465
I20260806 22:46:29.808414 10095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:42465 every 8 connection(s)
I20260806 22:46:29.808792 10025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:29.809674 10096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:29.812088 10096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b: Bootstrap starting.
I20260806 22:46:29.812661 10096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:29.812912 10096 log.cc:824] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:29.813642 10096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b: No bootstrap required, opened a new log
I20260806 22:46:29.815284 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 10025
I20260806 22:46:29.815407 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:29.815331 10096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f9b2a51fb44df3a5d9af00f6cb916b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42465 } }
I20260806 22:46:29.815537 10096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:29.815610 10096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f9b2a51fb44df3a5d9af00f6cb916b, State: Initialized, Role: FOLLOWER
I20260806 22:46:29.815698 19755 external_mini_cluster.cc:1428] Setting key 2f72a0bad1f0c9bb12aee4df2881006b
I20260806 22:46:29.815755 10096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f9b2a51fb44df3a5d9af00f6cb916b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42465 } }
I20260806 22:46:29.815853 10096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:29.815917 10096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:29.815972 10096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:29.817152 10096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f9b2a51fb44df3a5d9af00f6cb916b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42465 } }
I20260806 22:46:29.817250 10096 leader_election.cc:302] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53f9b2a51fb44df3a5d9af00f6cb916b; no voters:
I20260806 22:46:29.817435 10096 leader_election.cc:288] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:29.817483 10101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:29.817603 10101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [term 1 LEADER]: Becoming Leader. State: Replica: 53f9b2a51fb44df3a5d9af00f6cb916b, State: Running, Role: LEADER
I20260806 22:46:29.817720 10096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:29.817747 10101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f9b2a51fb44df3a5d9af00f6cb916b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42465 } }
I20260806 22:46:29.818073 10101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53f9b2a51fb44df3a5d9af00f6cb916b. Latest consensus state: current_term: 1 leader_uuid: "53f9b2a51fb44df3a5d9af00f6cb916b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f9b2a51fb44df3a5d9af00f6cb916b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42465 } } }
I20260806 22:46:29.818182 10101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:29.818084 10102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53f9b2a51fb44df3a5d9af00f6cb916b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f9b2a51fb44df3a5d9af00f6cb916b" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 42465 } } }
I20260806 22:46:29.818295 10102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:29.818579 10108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:29.819167 10108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:29.820823 10108 catalog_manager.cc:1347] Generated new cluster ID: 5f5c93d6c86444b0b0b11c5782e6c4ae
I20260806 22:46:29.820891 10108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:29.821579 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:29.816171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37961 (local address 127.19.74.254:42465)
0806 22:46:29.816362 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:29.816372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:29.816407 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:46:29.816518 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:29.816523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:29.816562 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:29.816666 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:29.816947 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:29.818128 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:29.819358 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:29.819974 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:29.820082 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:29.820857 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:29.820861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:29.820861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:29.821074 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:29.821107 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:29.821120 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:29.821205 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:29.821280 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:46:29.821363 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260806 22:46:29.822486 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:29.833504 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:29.823847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60061 (local address 127.19.74.254:42465)
0806 22:46:29.823882 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:29.823884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:29.830778 (+ 6894us) server_negotiation.cc:407] Connection header received
0806 22:46:29.830846 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:29.830847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:29.830859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:29.830915 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:29.831051 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:29.831518 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:29.832246 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:29.832449 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:29.832536 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:29.833212 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:29.833213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:29.833214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:29.833219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:29.833234 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:29.833238 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:29.833306 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:29.833367 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:46:29.833390 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:46:29.843247 10108 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:29.843883 10108 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:29.847374 10108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53f9b2a51fb44df3a5d9af00f6cb916b: Generated new TSK 0
I20260806 22:46:29.847553 10108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:29.851639 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 10025
2026-08-06T22:46:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:29 dist-test-slave-pgkr krb5kdc[10129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:29 dist-test-slave-pgkr krb5kdc[10129](info): set up 2 sockets
Aug 06 22:46:29 dist-test-slave-pgkr krb5kdc[10129](info): commencing operation
krb5kdc: starting...
W20260806 22:46:30.904810 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:30 dist-test-slave-pgkr krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:31.024987 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34535
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:34535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:31.109283 10143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:31.109529 10143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:31.109570 10143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:31.110754 10143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:31.110827 10143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:31.110850 10143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:31.110870 10143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:31.110901 10143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:31.112401 10143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:34535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:34535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:31.112694 10143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:31.112931 10143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:31.117950 10151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:31.117923 10148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:31.117978 10149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:31.118275 10143 server_base.cc:1034] running on GCE node
I20260806 22:46:31.118570 10143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:31.118949 10143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:31.120152 10143 hybrid_clock.cc:648] HybridClock initialized: now 1786056391120126 us; error 35 us; skew 500 ppm
Aug 06 22:46:31 dist-test-slave-pgkr krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056391, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:31.128178 10143 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:31.128747 10143 webserver.cc:460] Webserver started at http://127.19.74.254:39661/ using document root <none> and password file <none>
I20260806 22:46:31.128999 10143 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:31.129078 10143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:31.129243 10143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:31.130431 10143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "dd3f91b4b5654d38bb8deb48d2842deb"
format_stamp: "Formatted at 2026-08-06 22:46:31 on dist-test-slave-pgkr"
server_key: "7e533275e15809678a91dbb20bc65560"
server_key_iv: "c58f340813d3ef5b597e9216531cb017"
server_key_version: "encryptionkey@0"
I20260806 22:46:31.130863 10143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "dd3f91b4b5654d38bb8deb48d2842deb"
format_stamp: "Formatted at 2026-08-06 22:46:31 on dist-test-slave-pgkr"
server_key: "7e533275e15809678a91dbb20bc65560"
server_key_iv: "c58f340813d3ef5b597e9216531cb017"
server_key_version: "encryptionkey@0"
I20260806 22:46:31.132298 10143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:31.133119 10159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:31.133306 10143 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:31.133428 10143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "dd3f91b4b5654d38bb8deb48d2842deb"
format_stamp: "Formatted at 2026-08-06 22:46:31 on dist-test-slave-pgkr"
server_key: "7e533275e15809678a91dbb20bc65560"
server_key_iv: "c58f340813d3ef5b597e9216531cb017"
server_key_version: "encryptionkey@0"
I20260806 22:46:31.133531 10143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:31.137252 10143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:31.139210 10143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:31.139389 10143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:31.143663 10143 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:34535
I20260806 22:46:31.143709 10214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:34535 every 8 connection(s)
I20260806 22:46:31.144064 10143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:31.145059 10215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:31.147334 10215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb: Bootstrap starting.
I20260806 22:46:31.147828 10215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:31.148044 10215 log.cc:824] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:31.148666 10215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb: No bootstrap required, opened a new log
I20260806 22:46:31.150327 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 10143
I20260806 22:46:31.150435 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:31.150311 10215 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f91b4b5654d38bb8deb48d2842deb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34535 } }
I20260806 22:46:31.150473 10215 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:31.150511 10215 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd3f91b4b5654d38bb8deb48d2842deb, State: Initialized, Role: FOLLOWER
I20260806 22:46:31.150692 19755 external_mini_cluster.cc:1428] Setting key 5479185fcb72234da0bbf19821ec7f4a
I20260806 22:46:31.150696 10215 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f91b4b5654d38bb8deb48d2842deb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34535 } }
I20260806 22:46:31.150776 10215 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:31.150820 10215 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:31.150875 10215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:31.151520 10215 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f91b4b5654d38bb8deb48d2842deb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34535 } }
I20260806 22:46:31.151615 10215 leader_election.cc:302] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd3f91b4b5654d38bb8deb48d2842deb; no voters:
I20260806 22:46:31.151790 10215 leader_election.cc:288] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:31.151932 10220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:31.152070 10215 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:31.152094 10220 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [term 1 LEADER]: Becoming Leader. State: Replica: dd3f91b4b5654d38bb8deb48d2842deb, State: Running, Role: LEADER
I20260806 22:46:31.152544 10220 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f91b4b5654d38bb8deb48d2842deb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34535 } }
I20260806 22:46:31.153160 10221 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd3f91b4b5654d38bb8deb48d2842deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f91b4b5654d38bb8deb48d2842deb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34535 } } }
I20260806 22:46:31.153271 10221 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:31.153641 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:31.151186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54619 (local address 127.19.74.254:34535)
0806 22:46:31.151800 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:31.151809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:31.151848 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:46:31.151988 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:31.151995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:31.152025 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:31.152199 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:31.152948 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:31.152952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:31.152953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:31.152958 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:31.153006 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:31.153025 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:31.153091 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:31.153352 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 22:46:31.153401 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":523,"threads_started":1}
I20260806 22:46:31.153638 10233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:31.153625 10222 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd3f91b4b5654d38bb8deb48d2842deb. Latest consensus state: current_term: 1 leader_uuid: "dd3f91b4b5654d38bb8deb48d2842deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f91b4b5654d38bb8deb48d2842deb" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 34535 } } }
I20260806 22:46:31.154273 10222 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:31.154294 10233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:31.154747 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:31.156663 10233 catalog_manager.cc:1347] Generated new cluster ID: f22f806eb7ca4f4ca1243f99179fac31
I20260806 22:46:31.156718 10233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:31.157694 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:31.156428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45649 (local address 127.19.74.254:34535)
0806 22:46:31.156462 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:31.156464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:31.156574 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:46:31.156685 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:31.156686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:31.156692 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:31.156745 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:31.157396 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:31.157398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:31.157398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:46:31.157415 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:31.157430 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:46:31.157435 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:31.157519 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:31.157621 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:46:31.157634 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:46:31.172101 10233 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:31.172318 10233 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:31.182641 10233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd3f91b4b5654d38bb8deb48d2842deb: Generated new TSK 0
I20260806 22:46:31.182823 10233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:46:31.192767 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 10143
2026-08-06T22:46:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1340 ms)
[----------] 18 tests from AuthTokenIssuingTest (23956 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:31 dist-test-slave-pgkr krb5kdc[10249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:31 dist-test-slave-pgkr krb5kdc[10249](info): set up 2 sockets
Aug 06 22:46:31 dist-test-slave-pgkr krb5kdc[10249](info): commencing operation
krb5kdc: starting...
W20260806 22:46:32.248430 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:32.373062 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:33037
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:37259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:33037,127.19.74.253:39291,127.19.74.252:40117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:32.458094 10263 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:46:32.458354 10263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:32.458403 10263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:32.458449 10263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:32.459673 10263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:32.459748 10263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:32.459779 10263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:32.459815 10263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:32.459847 10263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:32.461362 10263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:37259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:33037,127.19.74.253:39291,127.19.74.252:40117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:33037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:32.461673 10263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:32.461925 10263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:32.466964 10271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:32.467075 10263 server_base.cc:1034] running on GCE node
W20260806 22:46:32.466961 10268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:32.466966 10269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:32.467530 10263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:32.467864 10263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:32.469039 10263 hybrid_clock.cc:648] HybridClock initialized: now 1786056392469014 us; error 34 us; skew 500 ppm
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:32.471817 10263 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:32.472222 10263 webserver.cc:460] Webserver started at http://127.19.74.254:45689/ using document root <none> and password file <none>
I20260806 22:46:32.472429 10263 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:32.472472 10263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:32.472582 10263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:32.473610 10263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "afc5f9ace0be43c2b440d657a835fbb0"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "a9ea5b298ae5168748104f40fa5422e0"
server_key_iv: "26b97d093ce90c226a1325b3f773af3d"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.473942 10263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "afc5f9ace0be43c2b440d657a835fbb0"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "a9ea5b298ae5168748104f40fa5422e0"
server_key_iv: "26b97d093ce90c226a1325b3f773af3d"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.475397 10263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:32.476193 10278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:32.476374 10263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:32.476507 10263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "afc5f9ace0be43c2b440d657a835fbb0"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "a9ea5b298ae5168748104f40fa5422e0"
server_key_iv: "26b97d093ce90c226a1325b3f773af3d"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.476593 10263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:32.497534 10263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:32.499955 10263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:32.500139 10263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:32.504263 10263 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:33037
I20260806 22:46:32.504297 10333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:33037 every 8 connection(s)
I20260806 22:46:32.504698 10263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:32.505777 10334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:32.507436 10334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:32.508195 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 10263
I20260806 22:46:32.508317 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:32.508520 19755 external_mini_cluster.cc:1428] Setting key 83c07103a0cf3cad623a656ad07e08ca
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.74.253@KRBTEST.COM; defaulting to no policy
I20260806 22:46:32.517797 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.508141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:33037 (local address 127.0.0.1:42194)
0806 22:46:32.508321 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.508376 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.508946 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.509257 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.509260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.509552 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.509706 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.509712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.510747 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.510748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.511581 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.511586 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.512115 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.512116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.512197 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.512731 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.512743 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.514356 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.516224 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.516231 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.516233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.516432 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.516684 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.516685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.516685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.516777 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.517172 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.517177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.517259 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.517375 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.517622 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":99,"thread_start_us":103,"threads_started":1}
I20260806 22:46:32.517800 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.508176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.19.74.254:33037)
0806 22:46:32.508423 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.508430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.508464 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:46:32.509124 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.509126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.509146 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.509252 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.509814 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.510667 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.511692 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.512000 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.512081 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.514461 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.514475 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.514475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.514482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.516120 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.516529 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.516532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.516532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.516615 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.516883 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.516884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.516884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.517087 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.517181 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.517410 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.517550 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":89,"threads_started":1}
I20260806 22:46:32.518720 10334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:32.519093 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.518984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56260)
0806 22:46:32.519030 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.519072 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:46:32.519265 10279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
I20260806 22:46:32.519739 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.519640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56264)
0806 22:46:32.519683 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.519721 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.253@KRBTEST.COM" created.
W20260806 22:46:32.519924 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:39291: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab.
Entry for principal kudu/127.19.74.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab.
I20260806 22:46:32.535586 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.253:39291
--webserver_interface=127.19.74.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab
--principal=kudu/127.19.74.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:37259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:33037,127.19.74.253:39291,127.19.74.252:40117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:46:32.569154 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } attempt: 1
I20260806 22:46:32.569653 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.569509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56276)
0806 22:46:32.569587 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.569622 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:46:32.570001 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.569902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56286)
0806 22:46:32.569950 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.569981 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:46:32.570097 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:39291: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
W20260806 22:46:32.617532 10342 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:46:32.617751 10342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:32.617780 10342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:32.617823 10342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:32.618995 10342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:32.619055 10342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:32.619079 10342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:32.619136 10342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:32.619168 10342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:32.620741 10342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:37259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:33037,127.19.74.253:39291,127.19.74.252:40117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab
--principal=kudu/127.19.74.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.253:39291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.74.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:32.621026 10342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:32.621255 10342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:32.626159 10348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:32.626135 10351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:32.626144 10349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:32.626176 10342 server_base.cc:1034] running on GCE node
I20260806 22:46:32.626688 10342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:32.627004 10342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:32.628178 10342 hybrid_clock.cc:648] HybridClock initialized: now 1786056392628173 us; error 43 us; skew 500 ppm
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:32.630729 10342 init.cc:377] Logged in from keytab as kudu/127.19.74.253@KRBTEST.COM (short username kudu)
I20260806 22:46:32.631153 10342 webserver.cc:460] Webserver started at http://127.19.74.253:46265/ using document root <none> and password file <none>
I20260806 22:46:32.631413 10342 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:32.631481 10342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:32.631651 10342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:32.632699 10342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/data/instance:
uuid: "ecf0890bbaab4849895fb6706d43ad2c"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "865b57589e73193161434c3caacab39f"
server_key_iv: "314c0a69d71d000899504095cc5fb168"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.633064 10342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/wal/instance:
uuid: "ecf0890bbaab4849895fb6706d43ad2c"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "865b57589e73193161434c3caacab39f"
server_key_iv: "314c0a69d71d000899504095cc5fb168"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.634347 10342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:32.635131 10358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:32.635325 10342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:32.635433 10342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/wal
uuid: "ecf0890bbaab4849895fb6706d43ad2c"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "865b57589e73193161434c3caacab39f"
server_key_iv: "314c0a69d71d000899504095cc5fb168"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.635509 10342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:32.638296 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } attempt: 2
I20260806 22:46:32.638707 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.638545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56292)
0806 22:46:32.638625 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.638667 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:46:32.639065 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.638978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56298)
0806 22:46:32.639013 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.639040 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 22:46:32.639184 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:39291: Network error: Client connection negotiation failed: client connection to 127.19.74.253:39291: connect: Connection refused (error 111)
I20260806 22:46:32.650331 10342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:32.652056 10342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:32.652212 10342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:32.656176 10342 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.253:39291
I20260806 22:46:32.656209 10412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.253:39291 every 8 connection(s)
I20260806 22:46:32.656572 10342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
I20260806 22:46:32.657573 10413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:32.659160 10413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:32.660602 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 10342
I20260806 22:46:32.660727 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-1/wal/instance
I20260806 22:46:32.661008 19755 external_mini_cluster.cc:1428] Setting key ac717d72b459331b4b69661680e099b5
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:32.668444 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.659829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.19.74.254:33037)
0806 22:46:32.659886 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.659888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.660093 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:46:32.660716 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.660717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.660732 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.660798 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.661643 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.662096 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.663350 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.663550 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.663645 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.665629 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.665630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.665631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.665639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.667070 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.667463 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.667465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.667465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.667527 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.667795 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.667796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.667796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.667924 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.667999 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.668268 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.668358 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:46:32.668599 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.659799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:33037 (local address 127.0.0.1:42208)
0806 22:46:32.659981 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.660014 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.660582 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.660913 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.660918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.661291 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.661547 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.661552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.662194 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.662195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.663231 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.663234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.663681 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.663682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.663796 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.664270 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.664278 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.665531 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.667167 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.667174 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.667176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.667386 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.667593 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.667594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.667595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.667724 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.667991 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.667998 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.668135 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.668234 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.668377 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.19.74.252@KRBTEST.COM; defaulting to no policy
I20260806 22:46:32.669112 10413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.252@KRBTEST.COM" created.
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:46:32.676630 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.669459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56312 (local address 127.19.74.253:39291)
0806 22:46:32.669614 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.669618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.669634 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:46:32.669652 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.669653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.669663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.669722 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.669959 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.670769 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.671478 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.671825 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.671901 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.673693 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.673704 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.673705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.673710 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.675226 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.675673 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.675675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.675675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.675760 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.676000 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.676001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.676001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.676139 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.676200 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.676367 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.676516 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260806 22:46:32.676638 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.669378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56312)
0806 22:46:32.669445 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.669457 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.669506 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.669742 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.669742 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.669850 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.669898 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.669899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.670843 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.670844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.671390 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.671392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.671910 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.671911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.671976 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.672422 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.672426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.673592 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.675317 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.675318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.675319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.675582 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.675840 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.675841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.675842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.675916 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.676212 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.676213 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.676262 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.676341 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.676463 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:46:32.677342 10413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:32.677706 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.677562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:34956)
0806 22:46:32.677622 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.677675 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:46:32.677857 10362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
I20260806 22:46:32.678367 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.678219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:34958)
0806 22:46:32.678292 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.678328 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 22:46:32.678551 10413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:40117: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab.
Entry for principal kudu/127.19.74.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab.
I20260806 22:46:32.688872 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.252:40117
--webserver_interface=127.19.74.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab
--principal=kudu/127.19.74.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:37259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:33037,127.19.74.253:39291,127.19.74.252:40117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:46:32.727833 10413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } attempt: 1
I20260806 22:46:32.728360 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.728205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:34966)
0806 22:46:32.728277 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.728326 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:46:32.728724 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.728636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:34982)
0806 22:46:32.728678 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.728705 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:46:32.728818 10413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:40117: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
I20260806 22:46:32.730336 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } attempt: 3
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:46:32.737317 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.730688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56324)
0806 22:46:32.730737 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.730749 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.730812 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.730958 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.730959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.731132 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.731195 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.731196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.731861 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.731862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.732412 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.732414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.732822 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.732823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.732909 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.733388 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.733393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.734804 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.736245 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.736246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.736247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.736412 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.736624 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.736624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.736625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.736682 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.736963 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.736964 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.737009 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.737123 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.737223 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:46:32.737313 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.730726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.19.74.253:39291)
0806 22:46:32.730764 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.730766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.730794 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:46:32.730881 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.730882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.730893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.730947 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.731277 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.731758 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.732501 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.732747 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.732824 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.734886 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.734888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.734888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.734895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.736151 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.736482 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.736483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.736484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.736541 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.736767 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.736768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.736769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.736882 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.736973 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.737151 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.737222 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:46:32.737897 10334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:32.738967 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.738157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:34984)
0806 22:46:32.738861 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.738915 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":687}
I20260806 22:46:32.739298 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.739207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:34996)
0806 22:46:32.739253 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.739283 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:46:32.739385 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:40117: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
I20260806 22:46:32.770526 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } attempt: 1
I20260806 22:46:32.771026 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.770905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:35008)
0806 22:46:32.770963 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.770999 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:46:32.771404 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.771321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:35010)
0806 22:46:32.771367 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.771390 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:46:32.771487 10334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:40117: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
W20260806 22:46:32.773442 10420 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:46:32.773700 10420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:32.773737 10420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:32.773774 10420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:32.774971 10420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:32.775029 10420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:32.775055 10420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:32.775099 10420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:32.775135 10420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:32.776602 10420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:37259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:33037,127.19.74.253:39291,127.19.74.252:40117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab
--principal=kudu/127.19.74.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.252:40117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.74.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:32.776872 10420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:32.777088 10420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:32.781937 10426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:32.781929 10429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:32.781926 10427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:32.782202 10420 server_base.cc:1034] running on GCE node
I20260806 22:46:32.782429 10420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:32.782789 10420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:32.783951 10420 hybrid_clock.cc:648] HybridClock initialized: now 1786056392783946 us; error 39 us; skew 500 ppm
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:32.786319 10420 init.cc:377] Logged in from keytab as kudu/127.19.74.252@KRBTEST.COM (short username kudu)
I20260806 22:46:32.786777 10420 webserver.cc:460] Webserver started at http://127.19.74.252:34375/ using document root <none> and password file <none>
I20260806 22:46:32.786996 10420 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:32.787058 10420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:32.787210 10420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:32.788266 10420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/data/instance:
uuid: "46a41224df824dc2b203fd62434f76e8"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "a3dba312bdc0729207879da5addcb069"
server_key_iv: "3e449f36d1f2bafd1dfa500b21c9332b"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.788645 10420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/wal/instance:
uuid: "46a41224df824dc2b203fd62434f76e8"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "a3dba312bdc0729207879da5addcb069"
server_key_iv: "3e449f36d1f2bafd1dfa500b21c9332b"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.789983 10420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:32.790725 10438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:32.790897 10420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:32.790997 10420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/wal
uuid: "46a41224df824dc2b203fd62434f76e8"
format_stamp: "Formatted at 2026-08-06 22:46:32 on dist-test-slave-pgkr"
server_key: "a3dba312bdc0729207879da5addcb069"
server_key_iv: "3e449f36d1f2bafd1dfa500b21c9332b"
server_key_version: "encryptionkey@0"
I20260806 22:46:32.791078 10420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:32.796950 10413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } attempt: 2
I20260806 22:46:32.797326 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.797206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:35012)
0806 22:46:32.797263 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.797299 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:46:32.797662 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.797556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:35026)
0806 22:46:32.797600 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.797643 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:46:32.797747 10413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:40117: Network error: Client connection negotiation failed: client connection to 127.19.74.252:40117: connect: Connection refused (error 111)
I20260806 22:46:32.814213 10420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:32.815986 10420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:32.816159 10420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:32.820147 10420 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.252:40117
I20260806 22:46:32.820179 10493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.252:40117 every 8 connection(s)
I20260806 22:46:32.820523 10420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
I20260806 22:46:32.821635 10494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:32.823259 10494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:32.824389 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 10420
I20260806 22:46:32.824515 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056127658350-19755-0/minicluster-data/master-2/wal/instance
I20260806 22:46:32.824717 19755 external_mini_cluster.cc:1428] Setting key 89f1893897ea58b82dadb78f87f69a43
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:46:32.832978 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.823939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42222 (local address 127.19.74.254:33037)
0806 22:46:32.824033 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.824036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.824213 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:46:32.824854 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.824856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.824869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.824948 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.825631 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.826382 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.827636 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.827917 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.827995 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.830202 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.830204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.830204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.830212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.831629 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.832005 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.832006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.832006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.832068 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.832335 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.832336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.832336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.832456 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.832529 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.832804 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.832889 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:46:32.833163 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.823866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:33037 (local address 127.0.0.1:42222)
0806 22:46:32.824073 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.824121 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.824720 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.825027 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.825032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.825371 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.825553 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.825558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.826462 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.826463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.827558 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.827560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.828019 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.828020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.828131 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.828611 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.828621 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.830114 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.831704 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.831710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.831712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.831922 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.832136 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.832137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.832138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.832257 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.832531 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.832539 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.832669 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.832779 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.832922 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260806 22:46:32.833688 10494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:32.834549 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.825115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.19.74.252:40117)
0806 22:46:32.825302 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.825307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.825355 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:46:32.825426 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.825429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.825449 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.825541 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.826290 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.827515 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.828440 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.828902 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.828982 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.831556 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.831566 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.831566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.831582 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.833256 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.833645 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.833649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.833649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.833713 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.833904 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.833905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.833905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.834084 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.834237 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.834318 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.834416 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:32.840770 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.833959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:39291 (local address 127.0.0.1:56336)
0806 22:46:32.834011 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.834021 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.834065 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.834232 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.834232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.834408 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.834438 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.834439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.835336 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.835337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.835891 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.835894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.836272 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.836273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.836331 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.836737 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.836741 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.837945 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.839626 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.839627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.839628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.839832 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.840085 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.840086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.840087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.840155 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.840428 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.840429 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.840490 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.840566 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.840656 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:46:32.840818 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.834014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.19.74.253:39291)
0806 22:46:32.834055 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.834056 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.834066 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:32.834153 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.834155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.834164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.834219 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.834536 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.835214 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.836006 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.836204 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.836255 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.838046 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.838048 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.838048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.838055 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.839550 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.839936 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.839937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.839937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.840003 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.840220 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.840221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.840221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.840352 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.840429 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.840599 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.840690 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:46:32.841426 10494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:32.843005 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.835388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.19.74.254:33037)
0806 22:46:32.835423 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.835425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.835433 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:32.835451 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.835452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.835460 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.835520 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.835941 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.836510 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.837342 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.837516 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.837569 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.840268 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.840269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.840269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.840275 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.841995 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.842311 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.842312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.842312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.842369 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.842548 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.842548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.842549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.842708 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.842862 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.842899 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.842953 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:46:32.846647 10334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } attempt: 2
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:46:32.848019 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.841697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:35038)
0806 22:46:32.841725 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.841734 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.841769 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.841890 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.841890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.841992 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.842016 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.842017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.842805 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.842806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.843377 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.843379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.843800 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.843801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.843859 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.844154 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.844159 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.845266 (+ 1107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.847035 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.847036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.847036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.847200 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.847396 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.847398 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.847398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.847462 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.847734 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.847735 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.847774 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.847846 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.847934 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260806 22:46:32.848183 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.841731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.19.74.252:40117)
0806 22:46:32.841769 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.841771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.841779 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:32.841806 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.841807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.841816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.841884 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.842115 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.842740 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.843462 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.843728 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.843793 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.845348 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.845349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.845350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.845356 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.846963 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.847292 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.847293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.847294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.847339 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.847533 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.847534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.847534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.847662 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.847730 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.847887 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.848119 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":169,"server-negotiator.queue_time_us":22}
I20260806 22:46:32.849855 10494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8: Bootstrap starting.
I20260806 22:46:32.850566 10494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8: Neither blocks nor log segments found. Creating new log.
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:46:32.850941 10494 log.cc:824] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:32.851773 10494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8: No bootstrap required, opened a new log
I20260806 22:46:32.853205 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.843677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.19.74.253:39291)
0806 22:46:32.843709 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.843711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.843720 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:32.843742 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.843743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.843751 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.843800 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.844338 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.844751 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.845460 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.845669 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.845740 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.848443 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.848444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.848444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.848450 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.852009 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.852417 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.852418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.852418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.852472 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.852698 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.852699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.852700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.852807 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.852983 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.853019 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.853096 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:46:32.854040 10494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } }
I20260806 22:46:32.854241 10494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a41224df824dc2b203fd62434f76e8, State: Initialized, Role: FOLLOWER
I20260806 22:46:32.854329 10494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } }
I20260806 22:46:32.854387 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.846899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:35040)
0806 22:46:32.846943 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.846953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.846989 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.847250 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.847251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.847413 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.847450 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.847452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.848359 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.848360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.849023 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.849026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.849406 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.849407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.849466 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.849868 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.849873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.851288 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.853283 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.853285 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.853285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.853497 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.853704 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.853705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.853705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.853774 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.854107 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.854107 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.854171 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.854225 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.854293 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:46:32.854457 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.846960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.19.74.252:40117)
0806 22:46:32.847120 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.847122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.847129 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:46:32.847148 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.847148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.847157 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.847226 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.847532 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.848263 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.849123 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.849312 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.849402 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.851390 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.851391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.851391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.851399 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.853175 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.853588 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.853589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.853589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.853644 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.853847 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.853848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.853849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.854021 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.854111 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.854273 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.854351 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260806 22:46:32.854688 10494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:32.855270 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:32.856374 10334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0: Bootstrap starting.
I20260806 22:46:32.857175 10334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:32.857503 10334 log.cc:824] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:32.856909 10308 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "afc5f9ace0be43c2b440d657a835fbb0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42236
W20260806 22:46:32.856886 10513 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:32.858048 10513 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:32.858052 10502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.858278 10502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:32.858876 10334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0: No bootstrap required, opened a new log
I20260806 22:46:32.860201 10334 raft_consensus.cc:348] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } }
I20260806 22:46:32.860332 10334 raft_consensus.cc:729] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afc5f9ace0be43c2b440d657a835fbb0, State: Initialized, Role: FOLLOWER
I20260806 22:46:32.860410 10334 consensus_queue.cc:260] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } }
I20260806 22:46:32.860777 10308 raft_consensus.cc:482] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:46:32.858580 10420 master_runner.cc:186] Error getting master registration for 127.19.74.254:33037: OK, instance_id { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" instance_seqno: 1786056392503147 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:46:32.860855 10308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:32.860927 10420 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:46:32.861598 10308 raft_consensus.cc:504] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } }
I20260806 22:46:32.861845 10308 leader_election.cc:288] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [CANDIDATE]: Term 1 election: Requested vote from peers ecf0890bbaab4849895fb6706d43ad2c (127.19.74.253:39291), 46a41224df824dc2b203fd62434f76e8 (127.19.74.252:40117)
I20260806 22:46:32.862202 10466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afc5f9ace0be43c2b440d657a835fbb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "46a41224df824dc2b203fd62434f76e8"
I20260806 22:46:32.862313 10466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:32.863194 10466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afc5f9ace0be43c2b440d657a835fbb0 in term 1.
I20260806 22:46:32.863415 10282 leader_election.cc:302] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a41224df824dc2b203fd62434f76e8, afc5f9ace0be43c2b440d657a835fbb0; no voters:
I20260806 22:46:32.860729 10334 sys_catalog.cc:564] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:32.866242 10263 master_runner.cc:186] Error getting master registration for 127.19.74.253:39291: OK, instance_id { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" instance_seqno: 1786056392655028 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:46:32.869058 10263 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:46:32.860721 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.869257 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:32.869313 10516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:32.869357 10516 raft_consensus.cc:686] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [term 1 LEADER]: Becoming Leader. State: Replica: afc5f9ace0be43c2b440d657a835fbb0, State: Running, Role: LEADER
I20260806 22:46:32.869431 10516 consensus_queue.cc:237] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } }
I20260806 22:46:32.869855 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afc5f9ace0be43c2b440d657a835fbb0. Latest consensus state: current_term: 1 leader_uuid: "afc5f9ace0be43c2b440d657a835fbb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.869949 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: This master's current role is: LEADER
W20260806 22:46:32.870648 10536 catalog_manager.cc:1558] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:32.870700 10536 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:32.871276 10539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:32.871426 10539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:32.872100 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.863038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.19.74.253:39291)
0806 22:46:32.863141 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.863143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.863364 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:46:32.863471 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.863471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.863483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.863512 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.864327 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.865070 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.866041 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.866909 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.866957 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.868947 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.868949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.868949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.868956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.870731 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.871157 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.871159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.871159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.871220 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.871444 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.871445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.871447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.871581 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.871673 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.871908 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.871990 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:46:32.874583 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.863076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.19.74.252:40117)
0806 22:46:32.863109 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.863111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.863447 (+ 336us) server_negotiation.cc:407] Connection header received
0806 22:46:32.863669 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.863670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.863690 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.863719 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.864903 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.865609 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.867200 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.867554 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.867602 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.870506 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.870508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.870508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.870515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.872528 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.873466 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.873467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.873468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.873541 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.873844 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.873845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.873846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.873978 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.874058 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.874416 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.874497 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:46:32.875551 10466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [term 1 FOLLOWER]: Refusing update from remote peer afc5f9ace0be43c2b440d657a835fbb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:32.875990 10524 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:32.877332 10502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afc5f9ace0be43c2b440d657a835fbb0. Latest consensus state: current_term: 1 leader_uuid: "afc5f9ace0be43c2b440d657a835fbb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.877424 10502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:32.880452 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.869174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.19.74.254:33037)
0806 22:46:32.869211 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.869212 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.872781 (+ 3569us) server_negotiation.cc:407] Connection header received
0806 22:46:32.872881 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.872882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.872893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.872925 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.873518 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.874209 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.875182 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.875459 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.875498 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.877530 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.877532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.877532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.877539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.879202 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.879603 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.879604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.879604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.879663 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.879886 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.879887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.879887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.880017 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.880122 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.880245 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.880322 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:46:32.881206 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afc5f9ace0be43c2b440d657a835fbb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.881291 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:32.881389 10502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afc5f9ace0be43c2b440d657a835fbb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.881466 10502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:32.881649 10542 mvcc.cc:205] Tried to move back new op lower bound from 7315686985215488000 to 7315686985193906176. Current Snapshot: MvccSnapshot[applied={T|T < 7315686985215488000}]
I20260806 22:46:32.882889 10539 catalog_manager.cc:1347] Generated new cluster ID: 86ec0ef2173d40ef8f329a113b23a07b
I20260806 22:46:32.882930 10539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:32.888896 10413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } attempt: 3
Aug 06 22:46:32 dist-test-slave-pgkr krb5kdc[10249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056392, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:46:32.896895 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.889138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:40117 (local address 127.0.0.1:35068)
0806 22:46:32.889187 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:32.889197 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:32.889234 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:32.889363 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:32.889364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:32.889535 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:32.889580 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.889582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.890384 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.890385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.891200 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:32.891202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.891643 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.891644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:32.891706 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.892128 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:32.892133 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:32.893940 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:32.895856 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.895857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.895858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.896055 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.896277 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:32.896278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:32.896278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:32.896339 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:32.896607 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:32.896608 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:32.896662 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 22:46:32.896717 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:46:32.896805 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:46:32.897374 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:32.889181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.19.74.252:40117)
0806 22:46:32.889212 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.889213 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.889222 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:32.889283 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.889297 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.889307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.889360 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.889649 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.890326 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.891296 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:32.891579 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:32.891642 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:32.894026 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:32.894027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:32.894028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:32.894034 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:32.895754 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.896141 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.896142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.896142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.896204 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:32.896405 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:32.896405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:32.896406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:32.896526 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:32.896572 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:32.897228 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 22:46:32.897296 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:46:32.898927 10413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c: Bootstrap starting.
I20260806 22:46:32.899682 10413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:32.900002 10413 log.cc:824] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:32.900902 10413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c: No bootstrap required, opened a new log
I20260806 22:46:32.903048 10413 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } }
I20260806 22:46:32.903187 10413 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf0890bbaab4849895fb6706d43ad2c, State: Initialized, Role: FOLLOWER
I20260806 22:46:32.903278 10413 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } }
I20260806 22:46:32.903575 10548 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.903868 10548 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:32.903615 10413 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:32.903707 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afc5f9ace0be43c2b440d657a835fbb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ecf0890bbaab4849895fb6706d43ad2c"
I20260806 22:46:32.903683 10384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:32.905447 10384 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [term 1 FOLLOWER]: Refusing update from remote peer afc5f9ace0be43c2b440d657a835fbb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:32.905723 10516 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:32.906988 10385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afc5f9ace0be43c2b440d657a835fbb0 in term 1.
I20260806 22:46:32.908311 10548 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader afc5f9ace0be43c2b440d657a835fbb0. Latest consensus state: current_term: 1 leader_uuid: "afc5f9ace0be43c2b440d657a835fbb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.908447 10548 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:32.908516 10548 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afc5f9ace0be43c2b440d657a835fbb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.908599 10548 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:32.909677 10559 mvcc.cc:205] Tried to move back new op lower bound from 7315686985215488000 to 7315686985193906176. Current Snapshot: MvccSnapshot[applied={T|T < 7315686985215488000}]
I20260806 22:46:32.910333 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afc5f9ace0be43c2b440d657a835fbb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afc5f9ace0be43c2b440d657a835fbb0" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 33037 } } peers { permanent_uuid: "ecf0890bbaab4849895fb6706d43ad2c" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 39291 } } peers { permanent_uuid: "46a41224df824dc2b203fd62434f76e8" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 40117 } } }
I20260806 22:46:32.910423 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:32.911800 10539 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:32.911208 10563 catalog_manager.cc:1259] Loaded cluster ID: 86ec0ef2173d40ef8f329a113b23a07b
I20260806 22:46:32.913223 10563 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c: loading cluster ID for follower catalog manager: success
I20260806 22:46:32.914417 10563 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c: acquiring CA information for follower catalog manager: success
W20260806 22:46:32.914557 10563 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 22:46:32.915036 10539 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:32.923542 10539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afc5f9ace0be43c2b440d657a835fbb0: Generated new TSK 0
I20260806 22:46:32.923807 10539 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:46:32.954654 10337 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.953346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.19.74.254:33037)
0806 22:46:32.953387 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.953388 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.953815 (+ 427us) server_negotiation.cc:407] Connection header received
0806 22:46:32.954039 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.954040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.954054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.954086 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.954562 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42258: BlockingRecv error: recv got EOF from 127.0.0.1:42258 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:46:32.955327 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.953133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56374 (local address 127.19.74.253:39291)
0806 22:46:32.953180 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.953183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.953923 (+ 740us) server_negotiation.cc:407] Connection header received
0806 22:46:32.954872 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.954874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.954884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.954912 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.955266 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56374: BlockingRecv error: recv got EOF from 127.0.0.1:56374 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:46:32.956420 10500 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.955769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35078 (local address 127.19.74.252:40117)
0806 22:46:32.955810 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.955812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.956003 (+ 191us) server_negotiation.cc:407] Connection header received
0806 22:46:32.956027 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.956028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.956039 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.956083 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.956367 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35078: BlockingRecv error: recv got EOF from 127.0.0.1:35078 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:46:32.958775 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.957949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.19.74.253:39291)
0806 22:46:32.957982 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.957984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.958206 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:46:32.958306 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.958307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.958323 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.958353 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.958733 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56376: BlockingRecv error: recv got EOF from 127.0.0.1:56376 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:46:32.961683 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.960991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56384 (local address 127.19.74.253:39291)
0806 22:46:32.961021 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.961023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.961225 (+ 202us) server_negotiation.cc:407] Connection header received
0806 22:46:32.961313 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.961314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.961323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.961349 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.961644 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56384: BlockingRecv error: recv got EOF from 127.0.0.1:56384 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 22:46:32.965487 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.964704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.19.74.253:39291)
0806 22:46:32.964736 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.964737 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.965012 (+ 275us) server_negotiation.cc:407] Connection header received
0806 22:46:32.965119 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.965120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.965129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.965157 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.965445 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56390: BlockingRecv error: recv got EOF from 127.0.0.1:56390 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:46:32.971308 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.970549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.19.74.253:39291)
0806 22:46:32.970582 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.970583 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.970847 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:46:32.970935 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.970936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.970944 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.970971 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.971267 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56398: BlockingRecv error: recv got EOF from 127.0.0.1:56398 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:46:32.981247 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.980513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56402 (local address 127.19.74.253:39291)
0806 22:46:32.980545 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.980547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:32.980752 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:46:32.980846 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:32.980847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:32.980857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:32.980885 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:32.981194 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56402: BlockingRecv error: recv got EOF from 127.0.0.1:56402 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:46:33.000705 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:32.999773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56408 (local address 127.19.74.253:39291)
0806 22:46:32.999815 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:32.999817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:33.000065 (+ 248us) server_negotiation.cc:407] Connection header received
0806 22:46:33.000205 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:33.000206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:33.000219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:33.000251 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:33.000650 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56408: BlockingRecv error: recv got EOF from 127.0.0.1:56408 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:46:33.035279 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:33.034247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.19.74.253:39291)
0806 22:46:33.034295 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:33.034298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:33.034590 (+ 292us) server_negotiation.cc:407] Connection header received
0806 22:46:33.034734 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:33.034735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:33.034747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:33.034782 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:33.035214 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56424: BlockingRecv error: recv got EOF from 127.0.0.1:56424 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:46:33.101534 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:33.100777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.19.74.253:39291)
0806 22:46:33.100856 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:33.100859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:33.100971 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:46:33.101036 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:33.101037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:33.101047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:33.101090 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:33.101469 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56428: BlockingRecv error: recv got EOF from 127.0.0.1:56428 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 22:46:33.231765 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:33.230849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56434 (local address 127.19.74.253:39291)
0806 22:46:33.230927 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:33.230929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:33.231143 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:46:33.231199 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:33.231201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:33.231214 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:33.231263 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:33.231654 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56434: BlockingRecv error: recv got EOF from 127.0.0.1:56434 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 22:46:33.489619 10417 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:33.488946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56436 (local address 127.19.74.253:39291)
0806 22:46:33.489015 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:33.489017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:33.489039 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:46:33.489117 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:33.489118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:33.489129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:33.489178 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:33.489525 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56436: BlockingRecv error: recv got EOF from 127.0.0.1:56436 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:46:33.858603 10513 catalog_manager.cc:1259] Loaded cluster ID: 86ec0ef2173d40ef8f329a113b23a07b
I20260806 22:46:33.858734 10513 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8: loading cluster ID for follower catalog manager: success
I20260806 22:46:33.859560 10513 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8: acquiring CA information for follower catalog manager: success
I20260806 22:46:33.859876 10513 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 46a41224df824dc2b203fd62434f76e8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:46:33.915179 10563 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ecf0890bbaab4849895fb6706d43ad2c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:46:34.005506 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:34.002904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.19.74.253:39291)
0806 22:46:34.003026 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:34.003028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:34.003070 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:46:34.003152 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:34.003153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:34.003202 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:34.003288 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:34.003411 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.003973 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.004732 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.005018 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.005078 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:34.005222 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:34.005334 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:34.005389 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:34.005422 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:46:34.005438 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260806 22:46:34.009920 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:34.007445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35086 (local address 127.19.74.252:40117)
0806 22:46:34.007567 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:34.007569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:34.007577 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:34.007629 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:34.007630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:34.007665 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:34.007754 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:34.007889 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.008372 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.009105 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.009323 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.009392 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:34.009568 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:34.009681 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:34.009773 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:34.009812 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:46:34.009833 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260806 22:46:34.014739 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:34.011668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.19.74.254:33037)
0806 22:46:34.011765 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:34.011767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:34.011911 (+ 144us) server_negotiation.cc:407] Connection header received
0806 22:46:34.012003 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:34.012004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:34.012063 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:34.012154 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:34.012313 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.012844 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.013898 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.014096 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.014186 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:34.014379 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:34.014469 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:34.014554 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:34.014631 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:46:34.014653 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":51,"threads_started":1}
I20260806 22:46:34.014972 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:34.011627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56442 (local address 127.19.74.253:39291)
0806 22:46:34.011666 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:34.011668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:34.011959 (+ 291us) server_negotiation.cc:407] Connection header received
0806 22:46:34.011989 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:34.011990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:34.012002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:34.012066 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:34.012263 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.012995 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.013994 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.014287 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.014327 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:34.014536 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:34.014566 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:34.014589 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:34.014868 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:46:34.014902 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:46:34.015764 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:34.011710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35096 (local address 127.19.74.252:40117)
0806 22:46:34.011744 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:34.011746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:34.012074 (+ 328us) server_negotiation.cc:407] Connection header received
0806 22:46:34.012185 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:34.012186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:34.012197 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:34.012228 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:34.012424 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.013133 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.014882 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.015159 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.015206 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:34.015432 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:34.015463 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:34.015513 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:34.015634 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:46:34.015675 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:34.016588 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 10263
I20260806 22:46:34.024200 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 10342
I20260806 22:46:34.030689 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 10420
2026-08-06T22:46:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2839 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:34 dist-test-slave-pgkr krb5kdc[10648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:34 dist-test-slave-pgkr krb5kdc[10648](info): set up 2 sockets
Aug 06 22:46:34 dist-test-slave-pgkr krb5kdc[10648](info): commencing operation
krb5kdc: starting...
W20260806 22:46:35.071384 19755 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/test-user.keytab.
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.74.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
Entry for principal kudu/127.19.74.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab.
I20260806 22:46:35.189060 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38987
--webserver_interface=127.19.74.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38987,127.19.74.253:33515,127.19.74.252:39309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:35.272676 10663 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:46:35.272912 10663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:35.272943 10663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:35.272986 10663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:35.274135 10663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:35.274199 10663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:35.274221 10663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:35.274240 10663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:35.274269 10663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:35.275729 10663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38987,127.19.74.253:33515,127.19.74.252:39309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.254.keytab
--principal=kudu/127.19.74.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.254:38987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.74.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:35.276036 10663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:35.276271 10663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:35.281144 10671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.281186 10668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:35.281459 10663 server_base.cc:1034] running on GCE node
W20260806 22:46:35.281167 10669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:35.281749 10663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:35.282051 10663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:35.283236 10663 hybrid_clock.cc:648] HybridClock initialized: now 1786056395283204 us; error 57 us; skew 500 ppm
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:35.285698 10663 init.cc:377] Logged in from keytab as kudu/127.19.74.254@KRBTEST.COM (short username kudu)
I20260806 22:46:35.286123 10663 webserver.cc:460] Webserver started at http://127.19.74.254:39397/ using document root <none> and password file <none>
I20260806 22:46:35.286370 10663 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:35.286454 10663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:35.286603 10663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:35.287636 10663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/data/instance:
uuid: "134830a5344b4236a3885813920f8a16"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "ec0ed24035963130e25612b049957585"
server_key_iv: "97428f18713d307c30242fc4a3d277d2"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.287988 10663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/wal/instance:
uuid: "134830a5344b4236a3885813920f8a16"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "ec0ed24035963130e25612b049957585"
server_key_iv: "97428f18713d307c30242fc4a3d277d2"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.289266 10663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:35.289934 10679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.290114 10663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:35.290244 10663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/wal
uuid: "134830a5344b4236a3885813920f8a16"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "ec0ed24035963130e25612b049957585"
server_key_iv: "97428f18713d307c30242fc4a3d277d2"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.290318 10663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:35.304778 10663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:35.306466 10663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:35.306699 10663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:35.310742 10663 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.254:38987
I20260806 22:46:35.310801 10733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.254:38987 every 8 connection(s)
I20260806 22:46:35.311146 10663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:35.312116 10734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:35.313776 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 10663
I20260806 22:46:35.313655 10734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:35.313890 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-0/wal/instance
I20260806 22:46:35.314123 19755 external_mini_cluster.cc:1428] Setting key c624f86a1fbc1b1ac87c389a63bf5faf
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.74.253@KRBTEST.COM; defaulting to no policy
I20260806 22:46:35.323552 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.314331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38987 (local address 127.0.0.1:41112)
0806 22:46:35.314503 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.314537 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.315058 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.315386 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.315388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.315650 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.315786 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.315791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.316760 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.316761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.317421 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.317423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.317950 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.317952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.318012 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.318584 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.318591 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.320033 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.322042 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.322049 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.322051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.322262 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.322529 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.322530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.322530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.322647 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.323131 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.323138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.323194 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.323260 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.323376 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260806 22:46:35.323556 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.314530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.19.74.254:38987)
0806 22:46:35.314699 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.314703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.314734 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:46:35.315237 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.315240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.315269 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.315379 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.315882 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.316661 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.317525 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.317841 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.317946 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.320125 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.320134 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.320135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.320144 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.321961 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.322350 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.322353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.322354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.322458 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.322709 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.322710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.322710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.323043 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.323129 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.323325 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.323436 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260806 22:46:35.324277 10734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:35.324563 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.324486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48322)
0806 22:46:35.324512 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.324548 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 22:46:35.324718 10681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
I20260806 22:46:35.325136 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.325044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48330)
0806 22:46:35.325093 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.325120 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:46:35.325237 10734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:33515: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab.
Entry for principal kudu/127.19.74.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab.
I20260806 22:46:35.339134 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.253:33515
--webserver_interface=127.19.74.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab
--principal=kudu/127.19.74.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38987,127.19.74.253:33515,127.19.74.252:39309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:46:35.374439 10734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } attempt: 1
I20260806 22:46:35.374897 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.374753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48338)
0806 22:46:35.374829 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.374867 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:46:35.375293 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.375202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48342)
0806 22:46:35.375252 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.375276 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:46:35.375389 10734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:33515: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
W20260806 22:46:35.419431 10741 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:46:35.419638 10741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:35.419656 10741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:35.419703 10741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:35.420811 10741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:35.420848 10741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:35.420861 10741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:35.420895 10741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:35.420922 10741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:35.422336 10741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38987,127.19.74.253:33515,127.19.74.252:39309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.253.keytab
--principal=kudu/127.19.74.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.253:33515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.74.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:35.422573 10741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:35.422796 10741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:35.427619 10747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.427608 10750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.427587 10746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:35.427857 10741 server_base.cc:1034] running on GCE node
I20260806 22:46:35.428087 10741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:35.428370 10741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:35.429522 10741 hybrid_clock.cc:648] HybridClock initialized: now 1786056395429509 us; error 26 us; skew 500 ppm
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:35.431962 10741 init.cc:377] Logged in from keytab as kudu/127.19.74.253@KRBTEST.COM (short username kudu)
I20260806 22:46:35.432430 10741 webserver.cc:460] Webserver started at http://127.19.74.253:41257/ using document root <none> and password file <none>
I20260806 22:46:35.432598 10741 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:35.432639 10741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:35.432722 10741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:35.433755 10741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/data/instance:
uuid: "ad0f89b9a1204384a02215765c8563dc"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "9739ddd8dbb20a33cce2fd4539650317"
server_key_iv: "ca9549e5da0e373af0368d348db9bda3"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.434062 10741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/wal/instance:
uuid: "ad0f89b9a1204384a02215765c8563dc"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "9739ddd8dbb20a33cce2fd4539650317"
server_key_iv: "ca9549e5da0e373af0368d348db9bda3"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.435297 10741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:35.436017 10757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.436188 10741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.436292 10741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/wal
uuid: "ad0f89b9a1204384a02215765c8563dc"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "9739ddd8dbb20a33cce2fd4539650317"
server_key_iv: "ca9549e5da0e373af0368d348db9bda3"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.436364 10741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:35.443519 10734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } attempt: 2
I20260806 22:46:35.443878 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.443769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48348)
0806 22:46:35.443823 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.443854 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:46:35.444180 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.444092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48354)
0806 22:46:35.444133 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.444158 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:46:35.444264 10734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.253:33515: Network error: Client connection negotiation failed: client connection to 127.19.74.253:33515: connect: Connection refused (error 111)
I20260806 22:46:35.449991 10741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:35.452468 10741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:35.452621 10741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:35.456341 10741 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.253:33515
I20260806 22:46:35.456375 10809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.253:33515 every 8 connection(s)
I20260806 22:46:35.456696 10741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/data/info.pb
I20260806 22:46:35.457738 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:35.459296 10810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:35.463861 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 10741
I20260806 22:46:35.463981 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-1/wal/instance
I20260806 22:46:35.464195 19755 external_mini_cluster.cc:1428] Setting key bd13f7f2f1982019e6c8d76f134f293d
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:35.468829 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.459952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.19.74.254:38987)
0806 22:46:35.459996 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.459998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.460216 (+ 218us) server_negotiation.cc:407] Connection header received
0806 22:46:35.460719 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.460720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.460734 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.460783 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.461324 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.461764 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.462840 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.463056 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.463132 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.465621 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.465623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.465623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.465631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.467368 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.467871 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.467872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.467872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.467926 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.468224 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.468225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.468225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.468328 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.468382 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.468702 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.468770 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:46:35.469151 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.459911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38987 (local address 127.0.0.1:41128)
0806 22:46:35.460083 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.460115 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.460607 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.460844 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.460847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.461067 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.461233 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.461236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.461862 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.461863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.462737 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.462740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.463173 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.463175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.463297 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.463743 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.463752 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.465517 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.467467 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.467474 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.467476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.467787 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.467992 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.467993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.467993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.468157 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.468394 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.468401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.468567 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.468683 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.468853 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260806 22:46:35.469761 10810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.74.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.74.252@KRBTEST.COM" created.
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:46:35.479521 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.469994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48356)
0806 22:46:35.470205 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.470217 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.470266 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.470399 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.470400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.470543 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.470584 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.470585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.471439 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.471440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.471898 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.471899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.472404 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.472405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.472466 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.472887 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.472891 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.476674 (+ 3783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.478418 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.478419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.478419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.478594 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.478855 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.478855 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.478856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.478913 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.479227 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.479228 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.479265 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.479319 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.479417 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:46:35.479691 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.470078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.19.74.253:33515)
0806 22:46:35.470219 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.470224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.470266 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:46:35.470323 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.470325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.470337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.470400 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.470696 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.471343 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.471985 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.472315 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.472403 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.476760 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.476769 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.476769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.476777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.478310 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.478716 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.478720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.478720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.478777 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.478979 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.478979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.478980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.479133 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.479236 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.479359 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.479484 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260806 22:46:35.480226 10810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:35.480595 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.480466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:48972)
0806 22:46:35.480518 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.480568 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:46:35.480728 10761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
I20260806 22:46:35.481138 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.481015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:48976)
0806 22:46:35.481073 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.481110 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 22:46:35.481261 10810 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:39309: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.74.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab.
Entry for principal kudu/127.19.74.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab.
I20260806 22:46:35.489248 19755 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLdXImp/build/release/bin/kudu
/tmp/dist-test-taskLdXImp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.252:39309
--webserver_interface=127.19.74.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab
--principal=kudu/127.19.74.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.74.212:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.74.254:38987,127.19.74.253:33515,127.19.74.252:39309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:46:35.530499 10810 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } attempt: 1
I20260806 22:46:35.531126 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.530936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:48984)
0806 22:46:35.531013 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.531067 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:46:35.531579 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.531418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:48996)
0806 22:46:35.531517 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.531554 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 22:46:35.531731 10810 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:39309: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
I20260806 22:46:35.535404 10734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } attempt: 3
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:46:35.542454 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.535701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48372)
0806 22:46:35.535771 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.535785 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.535859 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.536026 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.536027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.536203 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.536266 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.536268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.536931 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.536932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.537442 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.537444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.537825 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.537826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.537909 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.538396 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.538400 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.539719 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.541530 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.541531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.541531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.541699 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.541906 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.541907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.541907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.541959 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.542227 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.542228 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.542266 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.542316 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.542390 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 22:46:35.542498 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.535768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.19.74.253:33515)
0806 22:46:35.535844 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.535846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.535856 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:35.535942 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.535943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.535953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.536020 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.536365 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.536846 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.537524 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.537749 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.537826 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.539855 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.539856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.539857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.539884 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.541437 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.541767 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.541769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.541769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.541836 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.542018 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.542019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.542019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.542162 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.542232 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.542340 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.542417 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:46:35.542934 10734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:35.543285 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.543169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49002)
0806 22:46:35.543221 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.543256 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:46:35.543606 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.543519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49016)
0806 22:46:35.543563 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.543589 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:46:35.543699 10734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:39309: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
W20260806 22:46:35.572134 10818 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:46:35.572351 10818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:35.572373 10818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:35.572393 10818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:35.573493 10818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:35.573534 10818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:35.573549 10818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:35.573561 10818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:35.573616 10818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:35.574841 10734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } attempt: 1
I20260806 22:46:35.575244 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.575117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49030)
0806 22:46:35.575177 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.575214 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:46:35.575237 10818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.74.212:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.74.254:38987,127.19.74.253:33515,127.19.74.252:39309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/krb5kdc/kudu_127.19.74.252.keytab
--principal=kudu/127.19.74.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.74.252:39309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.74.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:35.575482 10818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:35.575632 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.575519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49042)
0806 22:46:35.575572 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.575603 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:46:35.575748 10734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:39309: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
I20260806 22:46:35.575783 10818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:35.580291 10826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.580296 10823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.580302 10824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:35.580631 10818 server_base.cc:1034] running on GCE node
I20260806 22:46:35.580879 10818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:35.581164 10818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:35.582305 10818 hybrid_clock.cc:648] HybridClock initialized: now 1786056395582295 us; error 28 us; skew 500 ppm
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:35.584839 10818 init.cc:377] Logged in from keytab as kudu/127.19.74.252@KRBTEST.COM (short username kudu)
I20260806 22:46:35.585294 10818 webserver.cc:460] Webserver started at http://127.19.74.252:37789/ using document root <none> and password file <none>
I20260806 22:46:35.585512 10818 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:35.585577 10818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:35.585728 10818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:35.586791 10818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/data/instance:
uuid: "96747506b1184c86bca78abebab99744"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "5170683651140c0f849e03eeae020cc3"
server_key_iv: "0dec20ea9cf692c0f1b276f6cfd73d20"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.587157 10818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/wal/instance:
uuid: "96747506b1184c86bca78abebab99744"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "5170683651140c0f849e03eeae020cc3"
server_key_iv: "0dec20ea9cf692c0f1b276f6cfd73d20"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.588498 10818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:46:35.589211 10833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.589380 10818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.589485 10818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/data,/tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/wal
uuid: "96747506b1184c86bca78abebab99744"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-pgkr"
server_key: "5170683651140c0f849e03eeae020cc3"
server_key_iv: "0dec20ea9cf692c0f1b276f6cfd73d20"
server_key_version: "encryptionkey@0"
I20260806 22:46:35.589558 10818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:35.599905 10810 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } attempt: 2
I20260806 22:46:35.600271 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.600153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49056)
0806 22:46:35.600210 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.600244 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:46:35.600630 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.600533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49072)
0806 22:46:35.600576 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.600598 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:46:35.600726 10810 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.74.252:39309: Network error: Client connection negotiation failed: client connection to 127.19.74.252:39309: connect: Connection refused (error 111)
I20260806 22:46:35.602099 10818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:35.603967 10818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:35.604133 10818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:35.608119 10818 rpc_server.cc:307] RPC server started. Bound to: 127.19.74.252:39309
I20260806 22:46:35.608150 10887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.74.252:39309 every 8 connection(s)
I20260806 22:46:35.608479 10818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/data/info.pb
I20260806 22:46:35.609437 10888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:35.610944 10888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:35.613731 19755 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLdXImp/build/release/bin/kudu as pid 10818
I20260806 22:46:35.613850 19755 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLdXImp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056127658350-19755-0/minicluster-data/master-2/wal/instance
I20260806 22:46:35.614051 19755 external_mini_cluster.cc:1428] Setting key 7b5a421c7b3e2625aeb429c4842826e9
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:46:35.620708 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.611581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.19.74.254:38987)
0806 22:46:35.611667 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.611669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.611848 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:46:35.612375 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.612377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.612390 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.612442 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.613026 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.613497 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.614862 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.615170 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.615247 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.617634 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.617635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.617635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.617642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.619230 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.619637 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.619638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.619639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.619705 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.619958 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.619960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.619960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.620121 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.620192 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.620512 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.620577 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:46:35.620859 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.611543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.254:38987 (local address 127.0.0.1:41142)
0806 22:46:35.611729 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.611769 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.612242 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.612508 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.612512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.612752 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.612929 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.612934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.613596 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.613597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.614761 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.614763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.615278 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.615279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.615391 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.615860 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.615870 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.617543 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.619333 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.619340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.619341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.619549 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.619776 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.619777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.619777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.619881 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.620196 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.620204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.620358 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.620482 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.620641 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260806 22:46:35.621417 10888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:35.623649 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.614446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.19.74.252:39309)
0806 22:46:35.614595 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.614599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.614691 (+ 92us) server_negotiation.cc:407] Connection header received
0806 22:46:35.614760 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.614763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.614785 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.614855 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.615598 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.616664 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.617541 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.617913 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.617981 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.620368 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.620374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.620374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.620380 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.622435 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.622827 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.622829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.622830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.622882 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.623086 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.623087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.623087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.623259 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.623402 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.623456 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.623531 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:46:35.628906 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.621709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.253:33515 (local address 127.0.0.1:48388)
0806 22:46:35.621834 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.621844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.621896 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.622011 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.622011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.622217 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.622272 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.622274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.623107 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.623108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.623854 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.623856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.624274 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.624276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.624333 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.624737 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.624742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.626026 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.627929 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.627930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.627930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.628139 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.628343 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.628343 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.628344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.628423 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.628691 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.628691 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.628723 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.628774 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.628851 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":19}
I20260806 22:46:35.629086 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.621779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.19.74.253:33515)
0806 22:46:35.621846 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.621848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.621882 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:46:35.621954 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.621955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.621963 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.622007 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.622346 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.623024 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.623937 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.624200 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.624265 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.626136 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.626137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.626138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.626145 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.627844 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.628209 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.628210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.628211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.628273 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.628491 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.628492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.628492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.628626 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.628701 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.628819 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.628886 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.254@KRBTEST.COM
I20260806 22:46:35.629406 10888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:46:35.632480 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.624521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.19.74.254:38987)
0806 22:46:35.624575 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.624577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.624893 (+ 316us) server_negotiation.cc:407] Connection header received
0806 22:46:35.624922 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.624923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.624935 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.625022 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.625426 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.625929 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.626881 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.627213 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.627279 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.629763 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.629765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.629765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.629772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.631333 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.631671 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.631672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.631673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.631733 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.631926 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.631927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.631928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.632058 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.632231 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.632351 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.632422 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.252@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:46:35.636273 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.629632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49094)
0806 22:46:35.629679 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.629688 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.629722 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.629831 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.629832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.629982 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.630014 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.630016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.630836 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.630837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.631546 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.631548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.631895 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.631896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.631940 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.632344 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.632349 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.633681 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.635289 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.635290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.635290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.635440 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.635650 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.635651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.635651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.635715 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.635943 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.635944 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.635987 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.636068 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.636177 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:46:35.636413 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.629694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.19.74.252:39309)
0806 22:46:35.629732 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.629734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.629742 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:35.629764 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.629765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.629773 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.629822 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.630099 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.630735 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.631627 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.631827 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.631897 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.633759 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.633760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.633760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.633768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.635212 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.635508 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.635509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.635509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.635579 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.635774 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.635774 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.635775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.635869 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.635910 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.636103 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.636290 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":20}
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.74.253@KRBTEST.COM
I20260806 22:46:35.637677 10888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744: Bootstrap starting.
I20260806 22:46:35.638746 10888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:35.639071 10888 log.cc:824] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:35.639940 10888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744: No bootstrap required, opened a new log
I20260806 22:46:35.640885 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.633136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.19.74.253:33515)
0806 22:46:35.633505 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.633507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.633517 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:35.633537 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.633538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.633547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.633610 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.634207 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.634781 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.635635 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.635904 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.635981 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.638099 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.638100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.638100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.638108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.639808 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.640180 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.640182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.640182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.640242 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.640419 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.640420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.640420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.640540 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.640695 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.640732 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.640806 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328}
I20260806 22:46:35.642194 10888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } }
I20260806 22:46:35.642382 10888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96747506b1184c86bca78abebab99744, State: Initialized, Role: FOLLOWER
I20260806 22:46:35.642504 19755 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:46:35.642490 10888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } }
I20260806 22:46:35.642856 10888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:35.642830 10894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.642935 10894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:35.643699 10707 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "134830a5344b4236a3885813920f8a16"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41150
W20260806 22:46:35.644783 10906 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:35.644850 10906 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:35.645030 10818 master_runner.cc:186] Error getting master registration for 127.19.74.254:38987: OK, instance_id { permanent_uuid: "134830a5344b4236a3885813920f8a16" instance_seqno: 1786056395309553 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:46:35.647730 10818 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:46:35.650864 10734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } attempt: 2
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.254@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:46:35.658574 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.651196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.19.74.252:39309)
0806 22:46:35.651235 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.651251 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.651261 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:35.651343 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.651344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.651353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.651410 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.651751 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.652462 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.653446 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.653736 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.653797 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.655966 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.655968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.655968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.655975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.657407 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.657783 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.657785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.657785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.657842 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.658047 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.658048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.658048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.658184 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.658262 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.658409 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.658484 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:35.658574 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.651146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49108)
0806 22:46:35.651210 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.651224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.651275 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.651425 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.651427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.651606 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.651673 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.651675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.652548 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.652550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.653359 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.653362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.653813 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.653814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.653872 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.654354 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.654360 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.655880 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.657520 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.657521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.657522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.657700 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.657918 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.657918 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.657919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.657979 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.658255 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.658256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.658301 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.658385 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.658483 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:46:35.659914 10734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16: Bootstrap starting.
I20260806 22:46:35.660564 10734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:35.660859 10734 log.cc:824] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:35.661628 10734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16: No bootstrap required, opened a new log
I20260806 22:46:35.662712 10734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } }
I20260806 22:46:35.662848 10734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134830a5344b4236a3885813920f8a16, State: Initialized, Role: FOLLOWER
I20260806 22:46:35.662945 10734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } }
I20260806 22:46:35.663342 10734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:35.663300 10910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.663370 10707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:46:35.663414 10910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:35.663611 10707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:35.664535 10707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } }
W20260806 22:46:35.665473 10921 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:35.665544 10921 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:35.665655 10860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "134830a5344b4236a3885813920f8a16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "96747506b1184c86bca78abebab99744"
I20260806 22:46:35.665783 10860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:35.666667 10860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 134830a5344b4236a3885813920f8a16 in term 1.
I20260806 22:46:35.666934 10683 leader_election.cc:302] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 134830a5344b4236a3885813920f8a16, 96747506b1184c86bca78abebab99744; no voters:
I20260806 22:46:35.667064 10910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:35.667239 10910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [term 1 LEADER]: Becoming Leader. State: Replica: 134830a5344b4236a3885813920f8a16, State: Running, Role: LEADER
I20260806 22:46:35.667330 10910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } }
I20260806 22:46:35.667775 10707 leader_election.cc:288] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [CANDIDATE]: Term 1 election: Requested vote from peers ad0f89b9a1204384a02215765c8563dc (127.19.74.253:33515), 96747506b1184c86bca78abebab99744 (127.19.74.252:39309)
I20260806 22:46:35.668891 10923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 134830a5344b4236a3885813920f8a16. Latest consensus state: current_term: 1 leader_uuid: "134830a5344b4236a3885813920f8a16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.666327 10663 master_runner.cc:186] Error getting master registration for 127.19.74.253:33515: OK, instance_id { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" instance_seqno: 1786056395455340 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:46:35.669000 10923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:35.669041 10663 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:46:35.669219 10929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:35.669380 10929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:35.679241 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.669388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.19.74.252:39309)
0806 22:46:35.669428 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.669431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.670128 (+ 697us) server_negotiation.cc:407] Connection header received
0806 22:46:35.670249 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.670250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.670261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.670339 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.671131 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.671908 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.673554 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.673869 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.673909 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.675895 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.675896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.675897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.675905 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.677664 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.678304 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.678305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.678305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.678365 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.678566 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.678567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.678567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.678716 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.678799 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.679025 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.679116 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:46:35.679275 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.669713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.19.74.253:33515)
0806 22:46:35.669757 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.669759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.670094 (+ 335us) server_negotiation.cc:407] Connection header received
0806 22:46:35.670208 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.670210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.670222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.670255 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.671228 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.671985 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.673078 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.673870 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.673910 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.675689 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.675691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.675691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.675698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.677419 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.678093 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.678094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.678094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.678167 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.678503 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.678504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.678505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.678671 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.678747 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.679078 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.679170 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:35.681908 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.669934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.19.74.254:38987)
0806 22:46:35.669976 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.669978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.671057 (+ 1079us) server_negotiation.cc:407] Connection header received
0806 22:46:35.671305 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.671306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.671316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.671343 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.672057 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.672768 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.673776 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.674058 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.674095 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.678798 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.678800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.678800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.678808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.680353 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.680975 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.680976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.680977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.681037 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.681322 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.681323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.681324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.681459 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.681571 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.681749 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.681827 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:46:35.687460 10860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [term 1 FOLLOWER]: Refusing update from remote peer 134830a5344b4236a3885813920f8a16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:35.687815 10923 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:35.689244 10894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 134830a5344b4236a3885813920f8a16. Latest consensus state: current_term: 1 leader_uuid: "134830a5344b4236a3885813920f8a16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.689347 10894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:35.690750 10923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "134830a5344b4236a3885813920f8a16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.690840 10923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:35.691418 10894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "134830a5344b4236a3885813920f8a16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.691501 10894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:35.691751 10933 mvcc.cc:205] Tried to move back new op lower bound from 7315686996733370368 to 7315686996654092288. Current Snapshot: MvccSnapshot[applied={T|T < 7315686996733370368}]
I20260806 22:46:35.691857 10810 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } attempt: 3
I20260806 22:46:35.691934 10934 mvcc.cc:205] Tried to move back new op lower bound from 7315686996733370368 to 7315686996654092288. Current Snapshot: MvccSnapshot[applied={T|T < 7315686996733370368}]
I20260806 22:46:35.695955 10929 catalog_manager.cc:1347] Generated new cluster ID: 8be2c09b2a0c43619b8cf9e114ca9620
I20260806 22:46:35.696003 10929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:46:35 dist-test-slave-pgkr krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.74.253@KRBTEST.COM for kudu/127.19.74.252@KRBTEST.COM
I20260806 22:46:35.701538 10929 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:35.702780 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.692138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.74.252:39309 (local address 127.0.0.1:49132)
0806 22:46:35.692185 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.692196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.692231 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.692428 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.692429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.692640 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.692705 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.692707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.693777 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.693778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.694708 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.694711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.695250 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.695251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.695340 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.695879 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.695885 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.698206 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.700275 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.700276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.700277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.700502 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.701679 (+ 1177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.701680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.701681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.701766 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.702152 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.702152 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.702212 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.702276 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.702670 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:46:35.702940 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.692194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49132 (local address 127.19.74.252:39309)
0806 22:46:35.692255 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.692258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.692265 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:46:35.692307 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.692308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.692317 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.692377 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.692831 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.693607 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.694833 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.695129 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.695186 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.698304 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.698306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.698306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.698313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.700168 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.700990 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.700991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.700991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.701065 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.701850 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.701851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.701851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.702023 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.702094 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.702433 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.702520 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:46:35.703390 10929 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:35.704463 10810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc: Bootstrap starting.
I20260806 22:46:35.705229 10810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:35.705538 10810 log.cc:824] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:35.707005 10810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc: No bootstrap required, opened a new log
I20260806 22:46:35.709076 10810 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } }
I20260806 22:46:35.709235 10810 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad0f89b9a1204384a02215765c8563dc, State: Initialized, Role: FOLLOWER
I20260806 22:46:35.709318 10810 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } }
I20260806 22:46:35.709658 10810 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:35.709817 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "134830a5344b4236a3885813920f8a16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ad0f89b9a1204384a02215765c8563dc"
I20260806 22:46:35.709962 10784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:35.710870 10784 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 134830a5344b4236a3885813920f8a16 in term 1.
I20260806 22:46:35.711465 10783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [term 1 FOLLOWER]: Refusing update from remote peer 134830a5344b4236a3885813920f8a16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:35.712049 10910 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:35.717629 10929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16: Generated new TSK 0
I20260806 22:46:35.717846 10929 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:46:35.718221 10956 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:46:35.718286 10956 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:46:35.718397 10949 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "134830a5344b4236a3885813920f8a16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.718467 10949 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:35.721239 10949 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 134830a5344b4236a3885813920f8a16. Latest consensus state: current_term: 1 leader_uuid: "134830a5344b4236a3885813920f8a16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.721328 10949 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:35.721361 10949 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "134830a5344b4236a3885813920f8a16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.721398 10949 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:46:35.722039 10947 mvcc.cc:205] Tried to move back new op lower bound from 7315686996788350976 to 7315686996654092288. Current Snapshot: MvccSnapshot[applied={T|T < 7315686996733370368}]
I20260806 22:46:35.722800 10923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "134830a5344b4236a3885813920f8a16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "134830a5344b4236a3885813920f8a16" member_type: VOTER last_known_addr { host: "127.19.74.254" port: 38987 } } peers { permanent_uuid: "ad0f89b9a1204384a02215765c8563dc" member_type: VOTER last_known_addr { host: "127.19.74.253" port: 33515 } } peers { permanent_uuid: "96747506b1184c86bca78abebab99744" member_type: VOTER last_known_addr { host: "127.19.74.252" port: 39309 } } }
I20260806 22:46:35.722914 10923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134830a5344b4236a3885813920f8a16 [sys.catalog]: This master's current role is: LEADER
W20260806 22:46:35.738685 10892 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.737181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.19.74.252:39309)
0806 22:46:35.737227 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.737230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.737951 (+ 721us) server_negotiation.cc:407] Connection header received
0806 22:46:35.738085 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.738086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.738097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.738130 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.738573 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49140: BlockingRecv error: recv got EOF from 127.0.0.1:49140 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:46:35.739675 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.737647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.19.74.253:33515)
0806 22:46:35.737689 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.737691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.739120 (+ 1429us) server_negotiation.cc:407] Connection header received
0806 22:46:35.739234 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.739235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.739246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.739276 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.739600 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48414: BlockingRecv error: recv got EOF from 127.0.0.1:48414 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:46:35.740810 10737 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.740178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.19.74.254:38987)
0806 22:46:35.740231 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.740233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.740324 (+ 91us) server_negotiation.cc:407] Connection header received
0806 22:46:35.740399 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.740401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.740412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.740485 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.740745 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41162: BlockingRecv error: recv got EOF from 127.0.0.1:41162 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 22:46:35.747565 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.746870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.19.74.253:33515)
0806 22:46:35.746908 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.746910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.747028 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:46:35.747137 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.747138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.747157 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.747187 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.747519 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48422: BlockingRecv error: recv got EOF from 127.0.0.1:48422 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:46:35.750739 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.749946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.19.74.253:33515)
0806 22:46:35.749981 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.749982 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.750220 (+ 238us) server_negotiation.cc:407] Connection header received
0806 22:46:35.750316 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.750317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.750327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.750356 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.750699 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48438: BlockingRecv error: recv got EOF from 127.0.0.1:48438 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:46:35.754792 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.753952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48448 (local address 127.19.74.253:33515)
0806 22:46:35.753985 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.753987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.754215 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:46:35.754313 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.754314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.754324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.754352 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.754754 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48448: BlockingRecv error: recv got EOF from 127.0.0.1:48448 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:46:35.760700 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.759949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48454 (local address 127.19.74.253:33515)
0806 22:46:35.759981 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.759982 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.760201 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:46:35.760300 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.760301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.760311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.760339 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.760661 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48454: BlockingRecv error: recv got EOF from 127.0.0.1:48454 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:46:35.770673 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.769872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.19.74.253:33515)
0806 22:46:35.769905 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.769907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.770124 (+ 217us) server_negotiation.cc:407] Connection header received
0806 22:46:35.770219 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.770220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.770230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.770259 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.770573 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48464: BlockingRecv error: recv got EOF from 127.0.0.1:48464 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:46:35.789023 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.788090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.19.74.253:33515)
0806 22:46:35.788133 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.788135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.788406 (+ 271us) server_negotiation.cc:407] Connection header received
0806 22:46:35.788518 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.788519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.788531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.788564 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.788967 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48478: BlockingRecv error: recv got EOF from 127.0.0.1:48478 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:46:35.823436 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.822631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.19.74.253:33515)
0806 22:46:35.822681 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.822684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.822829 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:46:35.822918 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.822919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.822937 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.823008 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.823364 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48488: BlockingRecv error: recv got EOF from 127.0.0.1:48488 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:46:35.889243 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:35.888573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.19.74.253:33515)
0806 22:46:35.888653 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.888655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.888715 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:46:35.888769 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.888770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.888788 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.888842 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.889187 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48502: BlockingRecv error: recv got EOF from 127.0.0.1:48502 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 22:46:36.019287 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:36.018459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48514 (local address 127.19.74.253:33515)
0806 22:46:36.018535 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:36.018537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:36.018670 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:46:36.018770 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:36.018771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:36.018783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:36.018841 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:36.019201 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48514: BlockingRecv error: recv got EOF from 127.0.0.1:48514 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 22:46:36.277349 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:46:36.276655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.19.74.253:33515)
0806 22:46:36.276735 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:36.276738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:36.276749 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:36.276828 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:36.276830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:36.276849 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:36.276917 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:36.277301 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48528: BlockingRecv error: recv got EOF from 127.0.0.1:48528 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:46:36.645407 10906 catalog_manager.cc:1259] Loaded cluster ID: 8be2c09b2a0c43619b8cf9e114ca9620
I20260806 22:46:36.645524 10906 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744: loading cluster ID for follower catalog manager: success
I20260806 22:46:36.646373 10906 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744: acquiring CA information for follower catalog manager: success
I20260806 22:46:36.646719 10906 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 96747506b1184c86bca78abebab99744: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:46:36.718887 10956 catalog_manager.cc:1259] Loaded cluster ID: 8be2c09b2a0c43619b8cf9e114ca9620
I20260806 22:46:36.718973 10956 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc: loading cluster ID for follower catalog manager: success
I20260806 22:46:36.719853 10956 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc: acquiring CA information for follower catalog manager: success
I20260806 22:46:36.720199 10956 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ad0f89b9a1204384a02215765c8563dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:46:36.793313 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:36.790671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.19.74.253:33515)
0806 22:46:36.790815 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:36.790817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:36.790827 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:36.790855 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:36.790856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:36.790909 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:36.790997 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:36.791135 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.791700 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.792503 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.792730 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.792782 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:36.792979 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:36.793097 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:36.793212 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:36.793227 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 22:46:36.793245 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260806 22:46:36.797217 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:36.795093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.19.74.252:39309)
0806 22:46:36.795227 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:36.795229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:36.795236 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:46:36.795272 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:36.795273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:36.795307 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:36.795368 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:36.795430 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.795872 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.796500 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.796678 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.796724 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:36.796855 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:36.796941 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:36.796993 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:36.797106 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:46:36.797124 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260806 22:46:36.802003 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:36.799078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.19.74.253:33515)
0806 22:46:36.799128 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:36.799130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:36.799409 (+ 279us) server_negotiation.cc:407] Connection header received
0806 22:46:36.799508 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:36.799508 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:36.799518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:36.799573 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:36.799680 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.800225 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.801066 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.801398 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.801477 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:36.801681 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:36.801713 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:36.801789 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:36.801899 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:46:36.801922 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:46:36.802583 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:36.799216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.19.74.252:39309)
0806 22:46:36.799269 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:36.799271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:36.799450 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:46:36.799580 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:36.799581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:36.799595 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:36.799632 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:36.799862 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.800673 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.801483 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.801723 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.801798 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:36.801991 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:36.802020 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:36.802049 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:36.802467 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 22:46:36.802505 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:46:36.803220 11038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:36.799155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.19.74.254:38987)
0806 22:46:36.799352 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:36.799354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:36.799559 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:46:36.799616 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:36.799618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:36.799671 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:36.799770 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:36.799885 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.800733 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.801958 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:36.802314 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:36.802416 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:36.802814 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:36.802914 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:36.802983 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:36.803084 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:46:36.803110 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
I20260806 22:46:36.808471 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 10663
I20260806 22:46:36.816251 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 10741
I20260806 22:46:36.822007 19755 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLdXImp/build/release/bin/kudu with pid 10818
2026-08-06T22:46:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2791 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5630 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269172 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST